-- Logs begin at Mon 2022-09-05 13:39:09 UTC, end at Mon 2022-09-05 15:37:44 UTC. -- Sep 05 14:13:04.313179 dv-cinder-842905-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 05 14:13:04.323756 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 05 14:13:04.325793 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 5 14:13:04 2022] *** Sep 05 14:13:04.325793 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 05 14:13:04.325793 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 05 14:13:04.325793 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: nodename: dv-cinder-842905-3-iscsi Sep 05 14:13:04.325793 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: machine: x86_64 Sep 05 14:13:04.325793 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: clock source: unix Sep 05 14:13:04.325793 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: pcre jit disabled Sep 05 14:13:04.325793 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: detected number of CPU cores: 4 Sep 05 14:13:04.325793 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: current working directory: / Sep 05 14:13:04.325793 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: detected binary path: /usr/bin/uwsgi-core Sep 05 14:13:04.325793 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: your processes number limit is 47680 Sep 05 14:13:04.325793 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: your memory page size is 4096 bytes Sep 05 14:13:04.325793 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: detected max file descriptor number: 2048 Sep 05 14:13:04.326489 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: lock engine: pthread robust mutexes Sep 05 14:13:04.326489 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: thunder lock: enabled Sep 05 14:13:04.326489 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 05 14:13:04.326489 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 05 14:13:04.365608 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: Python main interpreter initialized at 0x561af58789c0 Sep 05 14:13:04.365608 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: python threads support enabled Sep 05 14:13:04.365608 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: your server socket listen backlog is limited to 100 connections Sep 05 14:13:04.365608 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: your mercy for graceful operations on workers is 90 seconds Sep 05 14:13:04.365931 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: mapped 403077 bytes (393 KB) for 2 cores Sep 05 14:13:04.366007 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: *** Operational MODE: preforking *** Sep 05 14:13:04.366217 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: *** uWSGI is running in multiple interpreter mode *** Sep 05 14:13:04.366217 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: spawned uWSGI master process (pid: 81583) Sep 05 14:13:04.366412 dv-cinder-842905-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 05 14:13:04.367572 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: spawned uWSGI worker 1 (pid: 81594, cores: 1) Sep 05 14:13:04.367572 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: spawned uWSGI worker 2 (pid: 81595, cores: 1) Sep 05 14:13:04.367572 dv-cinder-842905-3-iscsi devstack@keystone.service[81583]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 05 14:13:08.657271 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 05 14:13:08.657271 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: from cryptography.utils import int_from_bytes Sep 05 14:13:08.658077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 05 14:13:08.658077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: from cryptography.utils import int_from_bytes Sep 05 14:13:08.658823 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 05 14:13:08.658823 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: from cryptography.utils import int_from_bytes Sep 05 14:13:08.659292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 05 14:13:08.659292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: from cryptography.utils import int_from_bytes Sep 05 14:13:08.707396 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 05 14:13:08.707571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 05 14:13:08.707760 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] command line args: [] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 05 14:13:08.708031 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 05 14:13:08.708206 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ================================================================================ {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 05 14:13:08.708457 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] admin_token = **** {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.708701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] config_dir = [] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.708883 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 05 14:13:08.708971 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.709050 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] config_source = [] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.709126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 05 14:13:08.709226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] debug = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.709317 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] command line args: [] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 05 14:13:08.709424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.709675 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 05 14:13:08.709675 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ================================================================================ {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 05 14:13:08.709790 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.709929 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] admin_token = **** {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.710006 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.710184 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] config_dir = [] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.710268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.710370 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.710473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.710579 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] config_source = [] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.710676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] list_limit = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.710771 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] debug = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.710928 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] log_config_append = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.711042 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.711181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.711181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] log_dir = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.711366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.711461 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] log_file = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.711556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.711650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.711756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.711867 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.711938 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.712013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.712136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] list_limit = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.712217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] log_config_append = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.712295 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.712415 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.712489 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.712633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] log_dir = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.712709 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.713388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] log_file = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.713471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.713544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.713668 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.713790 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.713947 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.714073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.714187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.714307 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.714424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.714542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.714658 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.714778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.714920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.715044 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.715165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.715280 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.715400 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.715531 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.715531 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.715765 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.715887 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.716013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] publish_errors = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.716127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.716238 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.716357 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.716471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.716593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.716703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.716821 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.716976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.717094 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.717205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] standard_threads = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.717364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.717488 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.717602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] publish_errors = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.717602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.717839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.717839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.718051 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.718163 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] use_journal = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.718280 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.718403 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] use_json = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.718518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.718631 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] use_stderr = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.718631 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] use_syslog = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.718858 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.718982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.719138 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] standard_threads = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.719261 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.719376 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.719489 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.719602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.719602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.719827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.719942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] use_journal = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.720055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.720167 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] use_json = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.720285 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.720399 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] use_stderr = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.720399 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] use_syslog = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.720700 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.720819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 14:13:08.720940 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.721055 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.721233 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.721360 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.721475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.721589 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.721707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.721827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.721942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.721942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.722157 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.722270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.722382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.722495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.722615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.722736 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.722867 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.722990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.723111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.723226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.723357 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.723478 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.723592 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.723711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.723831 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.723944 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.724065 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.724179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.724291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.724404 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.724522 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.724636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.724761 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.724876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.724989 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.725108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.725228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.725343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.725463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.725583 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.725697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.725817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.725937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.726051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.726238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.726354 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.726468 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.726587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.726707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.726865 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.726993 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.727115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.727235 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.727350 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.727463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.727581 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.727703 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.727818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.727818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.728028 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.728140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.728140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.728378 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.728496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.728496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.728715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.728828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.728941 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.729063 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.729184 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.729304 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.729401 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.729496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.729589 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.729690 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.729786 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.729885 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.729885 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.730107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.730221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.730318 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.730434 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.730555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.730676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.730811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.730941 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.731065 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.731180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.731302 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.731417 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.731536 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.731660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.731777 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.731892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.732008 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.732124 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.732237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.732357 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.732471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.732593 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.732709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.732822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.732936 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.733050 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.733165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.733286 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.733402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.733515 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.733637 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.733753 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.733857 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.733980 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.734096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.734205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.734319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.734433 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.734554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.734677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.734845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.734963 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.735078 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.735194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.735308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.735434 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.735551 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.735671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.735793 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.735909 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.736023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.736144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.736261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.736375 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.736500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.736622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.736737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.736864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.736980 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.737094 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.737208 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.737324 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.737513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.737707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.737826 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.737941 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.737941 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.738165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.738282 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.738406 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.738528 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.738649 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.738770 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.738901 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.739028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.739154 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.739278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.739401 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.739517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.739643 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.739759 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.739880 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.740004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.740120 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.740235 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.740350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.740464 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.740578 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.740700 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.740816 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.740931 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.741056 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.741173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.741293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.741416 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.741533 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.741648 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.741762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.741876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.741991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.742112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.742112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.742329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.742444 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.742565 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.742688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.742842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.742961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.743076 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.743223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.743344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.743460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.743587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.743704 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.743818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.743932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.744047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.744161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] auth.external = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.744283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.744399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.744514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.744629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.744746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.744867 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.744990 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.745113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] auth.password = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.745234 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.745340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] auth.token = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.745452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.745597 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.745727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.745845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.745968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.746086 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.746200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.746316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.746430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.746544 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.746666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.746797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.746916 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.747032 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.747146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.747260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] credential.caching = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.747388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] auth.external = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.747547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.747666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.747782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.747782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.748357 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.748491 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.748610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.748610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] auth.password = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.748828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.748828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.749044 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] auth.token = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.749167 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.749284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.749399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.749520 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.749636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.749751 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.749883 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.750011 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.750128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.750250 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.750366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.750490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.750607 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.750722 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.750881 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.751001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] credential.caching = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.751117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.751245 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.751363 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] federation.caching = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.751478 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.751593 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.751708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.751831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.752019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.752145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.752262 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.752378 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.752494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.752609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.752733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.752850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.752850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.753067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.753182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.753297 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.753420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.753420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.753637 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.753752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.753866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.753981 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] identity.caching = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.753981 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.754206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.754322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.754322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.754538 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.754653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.754777 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.754910 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.754910 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.755127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.755242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.755369 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] federation.caching = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.755495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.755495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.755712 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.755828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.755828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.756042 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.756171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.756288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.756404 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.756519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.756634 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.756634 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.756857 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.756975 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.757091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.757207 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.757207 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.757424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.757548 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.757664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.757780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.757780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.758003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.758121 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.758261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.758378 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.758494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.758609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.758724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] identity.caching = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.758897 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.759030 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.759154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.759271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.759388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.759388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.760086 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.760212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.760330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.760447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.760562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.760677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.760800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.761151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.761273 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.761389 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.761505 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.761620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.761742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.761859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.761975 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.762091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.762206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.762322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.762444 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.762562 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.762678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.762806 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.762924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.763039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.763039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.763270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.763385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.763501 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.763616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.763732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.763855 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.763855 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.764083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.764199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.764199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.764414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.764537 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.764655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.764770 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.764770 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.764985 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.765100 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.765222 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.765339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.765454 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.765568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.765684 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.765799 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.765922 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.766039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.766153 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.766292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.766411 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.766534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.766750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.766905 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.767022 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.767168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.767289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.767455 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.767585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.767704 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.767819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.767934 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.768048 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.768163 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.768291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.768408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.768524 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.768638 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.768754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.768859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.768970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.769086 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.769207 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.769324 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.769439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.769554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.769677 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.769796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.769911 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.770026 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.770141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.770256 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.770379 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.770495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.770610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.770725 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.770859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.770978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.771105 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.771223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.771338 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.771453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.771567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.771682 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.771804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.771992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.772109 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.772224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.772339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.772454 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.772668 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.772788 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.772903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.773017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.773132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.773351 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] resource.caching = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.773477 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.773595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.773734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.773854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.774060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.774184 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.774309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.774471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.774590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.774762 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.774902 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.775031 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.775863 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.775992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.776108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.776265 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.776386 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] role.caching = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.776576 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] role.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.776694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.776810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.776984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.777104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.777255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.777382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.777500 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.777615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.777729 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.777865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.777985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.778108 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.778216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.778331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.778445 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.778588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.778707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.778879 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.778999 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.779115 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.779231 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.779351 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.779466 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.779590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.779707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.779823 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.779938 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.780053 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.780206 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.780336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.780453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.780569 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.780683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.780799 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.780913 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.781036 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.781152 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.781267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.781383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.781498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.781613 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.781772 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.781893 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.782081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.782199 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.782314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.782430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.782553 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.782737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.782881 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.782998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.783144 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.783264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.783476 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.783596 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.783713 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.783890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.784010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.784168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.784294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.784410 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.784599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.784718 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] token.caching = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.784834 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.784956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.785073 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.785262 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] resource.caching = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.785383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.785537 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.785655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.785783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.786002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.786122 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.786237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.786373 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.786492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.786705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.786876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.787036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.787156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.787271 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.787386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.787514 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.787633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.787749 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.787864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.787980 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.788094 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.788216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.788341 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] role.caching = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.788470 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.788587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] role.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.788703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.788817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.788932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.connection = **** {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.789047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.789170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.789287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.789409 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.789525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.789640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.789756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.789880 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.790000 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.790117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.790232 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.790347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.790462 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.790585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.790702 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.790860 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.790980 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.791095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.791211 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.791338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.791456 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.791571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.791687 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.791803 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.792230 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.792356 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.792356 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.792575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.792691 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.792806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.792932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.793049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.793165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.793280 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.793394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.793394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81594) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 05 14:13:08.793618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.793618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.793836 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.793952 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.794161 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.794405 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.794643 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.794913 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.795151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.795409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.795670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.795925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.796169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.796411 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.796649 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.796891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] token.caching = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.797136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.797391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.797634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.797896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.798151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.798397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.798646 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.798920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.799202 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.799445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.799689 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.799928 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.800181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.800463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.800700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.800948 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.801217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.801485 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.connection = **** {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.801733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.801986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.802226 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.802471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.802711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.802976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.803216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.803457 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.803708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.803949 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.804219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.804459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.804711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.804950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.805199 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.805433 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.805677 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.805916 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 14:13:08.806089 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81595) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 05 14:13:08.822917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:08.823148 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:08.823379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:08.831743 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:08.831868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:08.832069 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:08.832303 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:08.841475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:08.860104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:08.860331 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:08.861623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:08.861864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:08.913789 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:08.914618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:08.964614 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:08.964851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:08.965880 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:08.966116 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.121376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.121612 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.121742 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.121862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.122005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.122131 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.122257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.122375 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.122548 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.122677 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.122816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.122938 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.146645 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.146827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.146968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.147090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.147209 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.147327 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.147444 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.147561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.147687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.147806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.147956 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.148073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.148190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.148316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.158953 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.159087 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.171575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.171745 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.171878 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.171998 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.172116 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.172234 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.172352 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.172470 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.172597 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.172716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.172833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.172949 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.173066 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81594) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.173192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81595) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 14:13:09.190002 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.190211 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 05 14:13:09.190548 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.190669 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.190824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 05 14:13:09.190948 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 05 14:13:09.191067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.191183 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.191299 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.191417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 05 14:13:09.191547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.191667 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.191784 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.191900 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.192053 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.192313 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.201905 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.202183 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.202447 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.202570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.202725 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.202885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.203009 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.203291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.209696 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.210191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.212634 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.212964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.213131 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.213253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.213417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.213552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.213699 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.213822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.213981 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.214104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.214271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.214413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.214531 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.214619 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.214815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.214899 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.215093 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.215179 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.215361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.215442 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.215635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.215903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.237949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.238506 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.239441 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.239725 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.239987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.240272 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.244956 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.245236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.245320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.245513 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.245609 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.245868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.251873 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.252160 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.252353 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.252640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.256583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.256881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.257165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.257449 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.257717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.257818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.258015 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.258302 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.258587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.258898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.259182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.259456 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.260228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.260317 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.260522 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.260808 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.275853 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.276139 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.276349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.276629 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.280015 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.280510 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.283276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.283572 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.283656 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.283929 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.287767 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.288268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.291438 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.291928 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.294659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.294993 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.295130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.295276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.295430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.295565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.295715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.295843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.296001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.296132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.296396 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.296681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.312451 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.312737 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.313014 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.313166 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.313279 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.313367 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.313605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.314130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.320410 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.320923 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.321952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.322616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.323555 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.323841 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.324174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.324453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.326481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.326765 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.327072 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.327154 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.327354 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.327439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.327655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.327739 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.328092 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.328385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.338968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.339251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.339530 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.339648 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.339809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.339935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.340196 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.340279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.340488 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.340571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.340767 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.340849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.341039 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.341316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.356702 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.357185 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.359606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.360220 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.362487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.363102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.363954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.364525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.365469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.365751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.366006 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.366150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.366288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.366569 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.373096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.373683 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.374555 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.375154 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.377431 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.377828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.378038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.378135 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.378330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.378414 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.378614 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.378913 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.386214 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.386519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.386813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.386899 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.387129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.387426 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.396499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81594) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.396499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.396499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.396830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81595) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 14:13:09.396913 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.396998 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.397076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.397152 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.397227 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.397300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.397376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.397459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.397537 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.397612 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.397686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.397761 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.397850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.397929 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.398005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.398079 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.398155 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.398237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.398318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.398394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.398470 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81594) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.398544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.398544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.398544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81595) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 14:13:09.434774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561af58789c0 pid: 81594 (default app) Sep 05 14:13:09.434928 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561af58789c0 pid: 81595 (default app) Sep 05 14:13:09.436479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749fed7-f0b1-409a-99ad-6d1f239a7ecc None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:09.436766 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749fed7-f0b1-409a-99ad-6d1f239a7ecc None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:09.437041 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749fed7-f0b1-409a-99ad-6d1f239a7ecc None None] PATH_INFO: `/v3` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:09.441480 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Mon Sep 5 14:13:09 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:16.995963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af75328-5b86-409e-9f9f-2460642fdf6f None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:16.996475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af75328-5b86-409e-9f9f-2460642fdf6f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:16.996557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af75328-5b86-409e-9f9f-2460642fdf6f None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:17.000680 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:17.005940 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa42f74-8410-4c4e-96dd-0835ea7e6d0a None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:17.006405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa42f74-8410-4c4e-96dd-0835ea7e6d0a None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:17.006505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa42f74-8410-4c4e-96dd-0835ea7e6d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:17.077720 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: INFO dbcounter [None req-6fa42f74-8410-4c4e-96dd-0835ea7e6d0a None None] Registered counter for database keystone Sep 05 14:13:17.099260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG oslo_db.sqlalchemy.engines [None req-6fa42f74-8410-4c4e-96dd-0835ea7e6d0a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81594) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 05 14:13:17.105506 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writer thread running {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 05 14:13:17.288799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG passlib.handlers.bcrypt [None req-6fa42f74-8410-4c4e-96dd-0835ea7e6d0a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81594) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 05 14:13:17.289143 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG passlib.handlers.bcrypt [None req-6fa42f74-8410-4c4e-96dd-0835ea7e6d0a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81594) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 05 14:13:17.347740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-6fa42f74-8410-4c4e-96dd-0835ea7e6d0a None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:17.349347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-6fa42f74-8410-4c4e-96dd-0835ea7e6d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:17.478075 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 05 14:13:17.926676 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af75328-5b86-409e-9f9f-2460642fdf6f None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:17.996287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: INFO dbcounter [None req-8af75328-5b86-409e-9f9f-2460642fdf6f None None] Registered counter for database keystone Sep 05 14:13:18.014990 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG oslo_db.sqlalchemy.engines [None req-8af75328-5b86-409e-9f9f-2460642fdf6f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 05 14:13:18.021168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writer thread running {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 05 14:13:18.138685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af75328-5b86-409e-9f9f-2460642fdf6f None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:18.142903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b4b5f-5ab6-4943-8b9d-32397f36b44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:18.143536 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b4b5f-5ab6-4943-8b9d-32397f36b44f admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:18.143814 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b4b5f-5ab6-4943-8b9d-32397f36b44f admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:18.144082 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b4b5f-5ab6-4943-8b9d-32397f36b44f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:18.152701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b4b5f-5ab6-4943-8b9d-32397f36b44f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:18.204862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: WARNING keystone.common.rbac_enforcer.enforcer [None req-682b4b5f-5ab6-4943-8b9d-32397f36b44f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 05 14:13:18.205158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b4b5f-5ab6-4943-8b9d-32397f36b44f admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:18.214843 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Mon Sep 5 14:13:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 05 14:13:18.220819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa42f74-8410-4c4e-96dd-0835ea7e6d0a None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:18.235691 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa42f74-8410-4c4e-96dd-0835ea7e6d0a None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:18.240037 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d69a5-b513-4473-9dbc-803bb30ad85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:18.241137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d69a5-b513-4473-9dbc-803bb30ad85a admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:18.241413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d69a5-b513-4473-9dbc-803bb30ad85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:18.241680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d69a5-b513-4473-9dbc-803bb30ad85a admin admin] PATH_INFO: `/v3/projects/f189779a37444439ac0c4ef7b42cc427` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:18.243275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d69a5-b513-4473-9dbc-803bb30ad85a admin admin] RBAC: Authorizing `identity:get_project(project_id=f189779a37444439ac0c4ef7b42cc427)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:18.295915 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2e1d69a5-b513-4473-9dbc-803bb30ad85a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 05 14:13:18.296256 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d69a5-b513-4473-9dbc-803bb30ad85a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:18.300799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Mon Sep 5 14:13:18 2022] GET /identity/v3/projects/f189779a37444439ac0c4ef7b42cc427 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:20.162578 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4a89c-f4a1-4090-a4dc-ac90807c65f3 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:20.163116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4a89c-f4a1-4090-a4dc-ac90807c65f3 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:20.163219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4a89c-f4a1-4090-a4dc-ac90807c65f3 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:20.167253 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:13:20.172799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8bfe5-7212-4fe1-9770-0c61a8d74faa None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:20.173224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8bfe5-7212-4fe1-9770-0c61a8d74faa None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:20.173357 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8bfe5-7212-4fe1-9770-0c61a8d74faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:20.193987 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-afe8bfe5-7212-4fe1-9770-0c61a8d74faa None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:20.195583 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-afe8bfe5-7212-4fe1-9770-0c61a8d74faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:20.204051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 05 14:13:20.473293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4a89c-f4a1-4090-a4dc-ac90807c65f3 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:20.485863 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4a89c-f4a1-4090-a4dc-ac90807c65f3 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:20.490009 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfd89f-5b01-4ce8-8f5b-a1be8dce1407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:20.490606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfd89f-5b01-4ce8-8f5b-a1be8dce1407 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:20.490921 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfd89f-5b01-4ce8-8f5b-a1be8dce1407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:20.491191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfd89f-5b01-4ce8-8f5b-a1be8dce1407 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:20.498704 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfd89f-5b01-4ce8-8f5b-a1be8dce1407 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:20.500499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfd89f-5b01-4ce8-8f5b-a1be8dce1407 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:20.508999 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Mon Sep 5 14:13:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 14:13:20.514646 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8bfe5-7212-4fe1-9770-0c61a8d74faa None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:20.523638 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8bfe5-7212-4fe1-9770-0c61a8d74faa None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:20.528562 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473446c-dfa1-4aa0-aa89-8491329befdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:20.529188 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473446c-dfa1-4aa0-aa89-8491329befdc admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:20.529469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473446c-dfa1-4aa0-aa89-8491329befdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:20.529743 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473446c-dfa1-4aa0-aa89-8491329befdc admin admin] PATH_INFO: `/v3/users/40eedd635c074248bf2c6c67ebf16bb0` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:20.531074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473446c-dfa1-4aa0-aa89-8491329befdc admin admin] RBAC: Authorizing `identity:get_user(user_id=40eedd635c074248bf2c6c67ebf16bb0)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:20.532887 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473446c-dfa1-4aa0-aa89-8491329befdc admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:20.534396 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Mon Sep 5 14:13:20 2022] GET /identity/v3/users/40eedd635c074248bf2c6c67ebf16bb0 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:22.495906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ac426-3a56-4c01-8c60-49aac65507c1 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:22.496427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ac426-3a56-4c01-8c60-49aac65507c1 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:22.496522 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ac426-3a56-4c01-8c60-49aac65507c1 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:22.497782 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:13:22.502835 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8d559-9e7e-4d5e-b895-57a04848c150 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:22.503260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8d559-9e7e-4d5e-b895-57a04848c150 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:22.503389 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8d559-9e7e-4d5e-b895-57a04848c150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:22.521090 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-82a8d559-9e7e-4d5e-b895-57a04848c150 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:22.522264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-82a8d559-9e7e-4d5e-b895-57a04848c150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:22.552817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29532c6-d240-424f-a3aa-0d8300cd1d5e None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:22.553108 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29532c6-d240-424f-a3aa-0d8300cd1d5e None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:22.553388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29532c6-d240-424f-a3aa-0d8300cd1d5e None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:22.554703 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:13:22.559565 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1dd09-158f-4190-9751-ec71725f2a37 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:22.559852 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1dd09-158f-4190-9751-ec71725f2a37 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:22.560130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1dd09-158f-4190-9751-ec71725f2a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:22.601490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG passlib.handlers.bcrypt [None req-6cb1dd09-158f-4190-9751-ec71725f2a37 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81595) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 05 14:13:22.601876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG passlib.handlers.bcrypt [None req-6cb1dd09-158f-4190-9751-ec71725f2a37 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81595) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 05 14:13:22.606089 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:22.650665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-6cb1dd09-158f-4190-9751-ec71725f2a37 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:22.652793 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-6cb1dd09-158f-4190-9751-ec71725f2a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:22.750173 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:13:22.870744 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8d559-9e7e-4d5e-b895-57a04848c150 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:22.896249 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:22.974957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8d559-9e7e-4d5e-b895-57a04848c150 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:23.007258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a491ee-1045-41f1-9ff1-5003e28acfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:23.007819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a491ee-1045-41f1-9ff1-5003e28acfbb None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:23.008103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a491ee-1045-41f1-9ff1-5003e28acfbb None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:23.008382 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a491ee-1045-41f1-9ff1-5003e28acfbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:23.017161 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1dd09-158f-4190-9751-ec71725f2a37 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:23.019914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a491ee-1045-41f1-9ff1-5003e28acfbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:23.029498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-89a491ee-1045-41f1-9ff1-5003e28acfbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:13:23.029498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:23.029498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:23.029498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:23.029498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:23.029498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:23.029498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:23.029498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:23.029498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:23.029498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:23.029498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:23.029498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:23.029498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:23.029498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:23.029498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:23.029498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:23.030436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:23.030436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:23.030436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:23.030436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:23.030436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:23.030436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:23.030436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:23.030436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:23.030436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:23.030436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:23.030436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:23.030436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:23.030436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:23.030436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:23.030436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:23.030436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:23.031499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:23.031499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:23.031499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:23.031499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:23.031499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:23.031499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:23.031499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:23.031499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:23.031499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:23.031499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:23.031499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:13:23.031499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:13:23.031499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:13:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:23.036409 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a491ee-1045-41f1-9ff1-5003e28acfbb None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:23.118060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1dd09-158f-4190-9751-ec71725f2a37 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:23.119037 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:23.121686 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:23.138565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a491ee-1045-41f1-9ff1-5003e28acfbb None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:23.156214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a144c7c-acb6-45ce-95d6-55f5bad9d728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:23.157163 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a144c7c-acb6-45ce-95d6-55f5bad9d728 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:23.157446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a144c7c-acb6-45ce-95d6-55f5bad9d728 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:23.157719 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a144c7c-acb6-45ce-95d6-55f5bad9d728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:23.171637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298eb30-da58-4ae7-b6de-46a793be19cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:23.172240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298eb30-da58-4ae7-b6de-46a793be19cb None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:23.172521 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298eb30-da58-4ae7-b6de-46a793be19cb None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:23.172792 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298eb30-da58-4ae7-b6de-46a793be19cb None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:23.173449 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298eb30-da58-4ae7-b6de-46a793be19cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:23.173746 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298eb30-da58-4ae7-b6de-46a793be19cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:23.175162 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298eb30-da58-4ae7-b6de-46a793be19cb None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:23.187027 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:13:23.192853 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298eb30-da58-4ae7-b6de-46a793be19cb None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:23.224511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a144c7c-acb6-45ce-95d6-55f5bad9d728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:23.226056 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a144c7c-acb6-45ce-95d6-55f5bad9d728 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:23.292215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298eb30-da58-4ae7-b6de-46a793be19cb None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:23.324650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 8/16] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:13:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 14:13:23.326154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0116ea7-99e5-4e9c-acfb-5ecb6330e5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:23.326811 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0116ea7-99e5-4e9c-acfb-5ecb6330e5cc None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:23.327090 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0116ea7-99e5-4e9c-acfb-5ecb6330e5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:23.327366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0116ea7-99e5-4e9c-acfb-5ecb6330e5cc None admin] PATH_INFO: `/v3/users/40eedd635c074248bf2c6c67ebf16bb0` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:23.328502 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0116ea7-99e5-4e9c-acfb-5ecb6330e5cc None admin] RBAC: Authorizing `identity:get_user(user_id=40eedd635c074248bf2c6c67ebf16bb0)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:23.330005 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a144c7c-acb6-45ce-95d6-55f5bad9d728 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:23.330114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0116ea7-99e5-4e9c-acfb-5ecb6330e5cc None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:23.331779 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 9/17] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:23 2022] GET /identity/v3/users/40eedd635c074248bf2c6c67ebf16bb0 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:13:23.332926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:23.336930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0116ea7-99e5-4e9c-acfb-5ecb6330e5cc None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:23.351313 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:23.426327 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a144c7c-acb6-45ce-95d6-55f5bad9d728 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:23.433285 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0116ea7-99e5-4e9c-acfb-5ecb6330e5cc None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:23.460307 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1d766-f074-4df4-95ae-d33f383b64b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:23.461165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1d766-f074-4df4-95ae-d33f383b64b4 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:23.461446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1d766-f074-4df4-95ae-d33f383b64b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:23.461718 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1d766-f074-4df4-95ae-d33f383b64b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:23.466613 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e0b59-f9b7-4bc3-b2f0-1c9e0cd23d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:23.467278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e0b59-f9b7-4bc3-b2f0-1c9e0cd23d73 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:23.467565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e0b59-f9b7-4bc3-b2f0-1c9e0cd23d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:23.467845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e0b59-f9b7-4bc3-b2f0-1c9e0cd23d73 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:23.468966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e0b59-f9b7-4bc3-b2f0-1c9e0cd23d73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:23.470421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e0b59-f9b7-4bc3-b2f0-1c9e0cd23d73 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:23.472040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 10/18] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:13:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:13:23.477786 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e0b59-f9b7-4bc3-b2f0-1c9e0cd23d73 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:23.480085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1d766-f074-4df4-95ae-d33f383b64b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:23.481613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1d766-f074-4df4-95ae-d33f383b64b4 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:23.499917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-90c1d766-f074-4df4-95ae-d33f383b64b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:13:23.499917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:23.499917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:23.499917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:23.499917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:23.499917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:23.499917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:23.499917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:23.499917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:23.499917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:23.499917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:23.499917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:23.499917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:13:23.499917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:13:23.499917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:13:23.499917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:13:23.501309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:23.501309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:23.501309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:23.501309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:23.501309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:23.501309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:23.501309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:23.501309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:23.501309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:23.501309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:23.501309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:23.501309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:23.501309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:23.501309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:23.501309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:23.501309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:23.501309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:13:23.502116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:13:23.502116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:13:23.502116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:13:23.502116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 9/19] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:13:23.506662 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1d766-f074-4df4-95ae-d33f383b64b4 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:23.556696 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:23.562367 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:23.575125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e0b59-f9b7-4bc3-b2f0-1c9e0cd23d73 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:23.606935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1d766-f074-4df4-95ae-d33f383b64b4 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:23.607745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf30b0-eaf4-4899-8884-14beb961771d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:23.608463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf30b0-eaf4-4899-8884-14beb961771d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:23.608747 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf30b0-eaf4-4899-8884-14beb961771d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:23.609029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf30b0-eaf4-4899-8884-14beb961771d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:23.609746 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf30b0-eaf4-4899-8884-14beb961771d None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.domain.id=default, user.id=40eedd635c074248bf2c6c67ebf16bb0) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:23.610087 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf30b0-eaf4-4899-8884-14beb961771d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:23.611572 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf30b0-eaf4-4899-8884-14beb961771d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:23.617992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 11/20] 10.222.0.15 () {56 vars in 1322 bytes} [Mon Sep 5 14:13:23 2022] GET /identity/v3/role_assignments?user.id=40eedd635c074248bf2c6c67ebf16bb0&scope.domain.id=default&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 233 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:13:23.642349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e887f56-26b8-42a4-8cb6-3d59e5e23d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:23.643033 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e887f56-26b8-42a4-8cb6-3d59e5e23d77 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:23.643318 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e887f56-26b8-42a4-8cb6-3d59e5e23d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:23.643674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e887f56-26b8-42a4-8cb6-3d59e5e23d77 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:23.644623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e887f56-26b8-42a4-8cb6-3d59e5e23d77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:23.645117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e887f56-26b8-42a4-8cb6-3d59e5e23d77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:23.647120 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e887f56-26b8-42a4-8cb6-3d59e5e23d77 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:23.667217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 10/21] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:13:25.695839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ee6fd-c422-4ef0-b87d-94ed2015d3c0 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:25.696378 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ee6fd-c422-4ef0-b87d-94ed2015d3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:25.696378 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ee6fd-c422-4ef0-b87d-94ed2015d3c0 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:25.697982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 12/22] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:25.703251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe796db-de66-471f-a64d-1fdfec9b5e2f None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:25.704072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe796db-de66-471f-a64d-1fdfec9b5e2f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:25.704562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe796db-de66-471f-a64d-1fdfec9b5e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:25.729222 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-ebe796db-de66-471f-a64d-1fdfec9b5e2f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:25.730811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ebe796db-de66-471f-a64d-1fdfec9b5e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:25.795078 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde0bd4-1e05-4381-bb2a-363237e301aa None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:25.795685 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde0bd4-1e05-4381-bb2a-363237e301aa None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:25.796160 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde0bd4-1e05-4381-bb2a-363237e301aa None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:25.797829 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 13/23] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:13:25.802983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f3014-9e65-46ca-b265-de7f6603e128 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:25.803515 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f3014-9e65-46ca-b265-de7f6603e128 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:25.803992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f3014-9e65-46ca-b265-de7f6603e128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:25.828892 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 11/24] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:25.843774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-470f3014-9e65-46ca-b265-de7f6603e128 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:25.845281 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-470f3014-9e65-46ca-b265-de7f6603e128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:25.944657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 14/25] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:13:26.378267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe796db-de66-471f-a64d-1fdfec9b5e2f None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:26.427948 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:26.431239 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f3014-9e65-46ca-b265-de7f6603e128 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:26.563609 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe796db-de66-471f-a64d-1fdfec9b5e2f None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:26.574685 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:26.606226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f3014-9e65-46ca-b265-de7f6603e128 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:26.621136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eacd79c-f2cf-4053-b95d-5a3477887bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:26.622048 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacd79c-f2cf-4053-b95d-5a3477887bbf None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:26.622526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacd79c-f2cf-4053-b95d-5a3477887bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:26.623014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacd79c-f2cf-4053-b95d-5a3477887bbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:26.635887 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eacd79c-f2cf-4053-b95d-5a3477887bbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:26.643085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-2eacd79c-f2cf-4053-b95d-5a3477887bbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:13:26.643085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:26.643085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:26.643085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:26.643085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:26.643085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:26.643085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:26.643085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:26.643085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:26.643085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:26.643085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:26.643085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:26.643085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:26.643085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:26.643085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:26.643085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:26.643946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:26.643946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:26.643946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:26.643946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:26.643946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:26.643946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:26.643946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:26.643946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:26.643946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:26.643946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:26.643946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:26.643946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:26.643946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:26.643946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:26.643946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:26.643946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:26.644697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:26.644697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:26.644697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:26.644697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:26.644697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:26.644697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:26.644697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:26.644697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:26.644697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:26.644697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:26.644697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:13:26.644697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:13:26.645270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d7327-0966-4d53-bd41-f80abea38537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:26.645270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d7327-0966-4d53-bd41-f80abea38537 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:26.645270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d7327-0966-4d53-bd41-f80abea38537 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:26.645270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d7327-0966-4d53-bd41-f80abea38537 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:26.647400 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 12/26] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:13:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:26.648811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10e981-c011-40de-8e51-2fcccc42377c None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:26.649309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10e981-c011-40de-8e51-2fcccc42377c None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:26.649776 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10e981-c011-40de-8e51-2fcccc42377c None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:26.651312 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 13/27] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:26.652526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ec3f7-973b-466e-836c-0e9149c3b00e None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:26.653021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ec3f7-973b-466e-836c-0e9149c3b00e None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:26.653488 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ec3f7-973b-466e-836c-0e9149c3b00e None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:26.654878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 14/28] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:26.656712 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00c1b1-4d60-49fa-8b42-28116d6d0a28 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:26.657213 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00c1b1-4d60-49fa-8b42-28116d6d0a28 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:26.657681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00c1b1-4d60-49fa-8b42-28116d6d0a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:26.660398 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d7327-0966-4d53-bd41-f80abea38537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:26.662037 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d7327-0966-4d53-bd41-f80abea38537 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:26.677728 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-cd00c1b1-4d60-49fa-8b42-28116d6d0a28 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:26.679395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-cd00c1b1-4d60-49fa-8b42-28116d6d0a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:26.680268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-127d7327-0966-4d53-bd41-f80abea38537 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:13:26.680268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:26.680268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:26.680268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:26.680268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:26.680268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:26.680268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:26.680268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:26.680268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:26.680268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:26.680268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:26.680268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:26.680268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:13:26.680268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:13:26.680268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:13:26.680268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:13:26.681099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:26.681099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:26.681099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:26.681099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:26.681099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:26.681099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:26.681099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:26.681099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:26.681099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:26.681099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:26.681099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:26.681099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:26.681099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:26.681099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:26.681099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:26.681099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:26.681099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:13:26.681894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:13:26.681894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:13:26.681894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:13:26.681894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 15/29] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:13:26.682324 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d7327-0966-4d53-bd41-f80abea38537 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:26.756368 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 15/30] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:26.758434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f9808-5cb2-42c0-b2b0-13da8c289b4d None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:26.758898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f9808-5cb2-42c0-b2b0-13da8c289b4d None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:26.759314 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f9808-5cb2-42c0-b2b0-13da8c289b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:26.799839 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-cf0f9808-5cb2-42c0-b2b0-13da8c289b4d None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:26.800682 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d7327-0966-4d53-bd41-f80abea38537 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:26.801359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-cf0f9808-5cb2-42c0-b2b0-13da8c289b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:26.835160 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:26.840603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1d02b-8840-4db4-a98a-c2c58f19e74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:26.841492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1d02b-8840-4db4-a98a-c2c58f19e74a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:26.841972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1d02b-8840-4db4-a98a-c2c58f19e74a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:26.842447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1d02b-8840-4db4-a98a-c2c58f19e74a None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:26.843478 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf1d02b-8840-4db4-a98a-c2c58f19e74a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:26.844002 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf1d02b-8840-4db4-a98a-c2c58f19e74a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:26.845758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf1d02b-8840-4db4-a98a-c2c58f19e74a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:26.856276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 16/31] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:26.857494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1d02b-8840-4db4-a98a-c2c58f19e74a None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:26.877020 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:26.967172 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1d02b-8840-4db4-a98a-c2c58f19e74a None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:26.988114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 16/32] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:26.989306 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f9808-5cb2-42c0-b2b0-13da8c289b4d None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:27.004755 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186daf4-a353-4bdf-b5c3-0f4f078cd78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:27.005949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186daf4-a353-4bdf-b5c3-0f4f078cd78d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:27.006378 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186daf4-a353-4bdf-b5c3-0f4f078cd78d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:27.006825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186daf4-a353-4bdf-b5c3-0f4f078cd78d None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:27.007637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186daf4-a353-4bdf-b5c3-0f4f078cd78d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:27.008448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186daf4-a353-4bdf-b5c3-0f4f078cd78d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:27.010354 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186daf4-a353-4bdf-b5c3-0f4f078cd78d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:27.029926 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 17/33] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:27.035385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186daf4-a353-4bdf-b5c3-0f4f078cd78d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:27.102918 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:27.130545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f9808-5cb2-42c0-b2b0-13da8c289b4d None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:27.167922 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186daf4-a353-4bdf-b5c3-0f4f078cd78d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:27.170560 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be8c06-ebf4-49da-a1d4-90338deba3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:27.171486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be8c06-ebf4-49da-a1d4-90338deba3e5 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:27.171914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be8c06-ebf4-49da-a1d4-90338deba3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:27.172332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be8c06-ebf4-49da-a1d4-90338deba3e5 None admin] PATH_INFO: `/v3/users/40eedd635c074248bf2c6c67ebf16bb0` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:27.175031 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be8c06-ebf4-49da-a1d4-90338deba3e5 None admin] RBAC: Authorizing `identity:get_user(user_id=40eedd635c074248bf2c6c67ebf16bb0)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:27.176737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be8c06-ebf4-49da-a1d4-90338deba3e5 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:27.178711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 17/34] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:26 2022] GET /identity/v3/users/40eedd635c074248bf2c6c67ebf16bb0 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:27.183772 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be8c06-ebf4-49da-a1d4-90338deba3e5 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:27.209376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bece7f-a181-4b83-b695-46b2116ffb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:27.210419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bece7f-a181-4b83-b695-46b2116ffb06 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:27.210714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bece7f-a181-4b83-b695-46b2116ffb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:27.211023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bece7f-a181-4b83-b695-46b2116ffb06 None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:27.211833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bece7f-a181-4b83-b695-46b2116ffb06 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:27.213675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bece7f-a181-4b83-b695-46b2116ffb06 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:27.224074 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:27.300738 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be8c06-ebf4-49da-a1d4-90338deba3e5 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:27.338166 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d27615-68e8-4981-bf7f-e76d22b9c15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:27.339126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d27615-68e8-4981-bf7f-e76d22b9c15d None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:27.339585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d27615-68e8-4981-bf7f-e76d22b9c15d None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:27.340041 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d27615-68e8-4981-bf7f-e76d22b9c15d None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:27.340976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d27615-68e8-4981-bf7f-e76d22b9c15d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:27.349028 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d27615-68e8-4981-bf7f-e76d22b9c15d None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:27.555290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 18/35] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Sep 5 14:13:27 2022] POST /identity/v3/roles => generated 219 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:13:27.555290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75af39-92af-403f-831e-48bf83d00b49 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:27.555290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75af39-92af-403f-831e-48bf83d00b49 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:27.555290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75af39-92af-403f-831e-48bf83d00b49 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:27.556047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 19/36] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:27.558138 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75af39-92af-403f-831e-48bf83d00b49 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:27.593727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 18/37] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 5 14:13:27 2022] POST /identity/v3/projects => generated 301 bytes in 559 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:13:27.594814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bece7f-a181-4b83-b695-46b2116ffb06 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:27.722556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:27.733739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75af39-92af-403f-831e-48bf83d00b49 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:27.759019 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:27.763849 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bece7f-a181-4b83-b695-46b2116ffb06 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:27.781201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a098654-f573-435c-83c0-9d4a2688bc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:27.781855 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a098654-f573-435c-83c0-9d4a2688bc6e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:27.783968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a098654-f573-435c-83c0-9d4a2688bc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:27.783968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a098654-f573-435c-83c0-9d4a2688bc6e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:27.783968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a098654-f573-435c-83c0-9d4a2688bc6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:27.785275 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a098654-f573-435c-83c0-9d4a2688bc6e None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:27.789706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 20/38] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:13:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:27.789706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2910c-131e-4795-9030-6184a90cde91 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:27.789706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2910c-131e-4795-9030-6184a90cde91 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:27.789706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2910c-131e-4795-9030-6184a90cde91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:27.809565 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-35d2910c-131e-4795-9030-6184a90cde91 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:27.811351 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-35d2910c-131e-4795-9030-6184a90cde91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:27.814595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5365d-8183-41c2-9e0d-0f5b8af5ffc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:27.815264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5365d-8183-41c2-9e0d-0f5b8af5ffc3 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:27.815552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5365d-8183-41c2-9e0d-0f5b8af5ffc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:27.817754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5365d-8183-41c2-9e0d-0f5b8af5ffc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:27.817754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5365d-8183-41c2-9e0d-0f5b8af5ffc3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:27.818104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5365d-8183-41c2-9e0d-0f5b8af5ffc3 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:27.880994 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 21/39] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:27.882396 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85840f-72ff-40de-aec0-072147403a8b None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:27.882688 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85840f-72ff-40de-aec0-072147403a8b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:27.882997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85840f-72ff-40de-aec0-072147403a8b None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:27.888844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 22/40] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:27.888844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85840f-72ff-40de-aec0-072147403a8b None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:27.914090 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 19/41] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Sep 5 14:13:27 2022] POST /identity/v3/roles => generated 225 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:13:27.915408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348d3aa-bf6b-4c9a-8374-288f7361cd57 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:27.918301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348d3aa-bf6b-4c9a-8374-288f7361cd57 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:27.918301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348d3aa-bf6b-4c9a-8374-288f7361cd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:27.950209 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-8348d3aa-bf6b-4c9a-8374-288f7361cd57 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:27.952133 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8348d3aa-bf6b-4c9a-8374-288f7361cd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:28.009187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:28.020270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85840f-72ff-40de-aec0-072147403a8b None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:28.055754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 20/42] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:28.062701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363dc34-b5ac-4f80-9033-65b25109491a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:28.063349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363dc34-b5ac-4f80-9033-65b25109491a None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:28.063636 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363dc34-b5ac-4f80-9033-65b25109491a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:28.063918 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363dc34-b5ac-4f80-9033-65b25109491a None admin] PATH_INFO: `/v3/domains/default/users/40eedd635c074248bf2c6c67ebf16bb0/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:28.066114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363dc34-b5ac-4f80-9033-65b25109491a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=40eedd635c074248bf2c6c67ebf16bb0, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:28.067641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363dc34-b5ac-4f80-9033-65b25109491a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:28.162084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348d3aa-bf6b-4c9a-8374-288f7361cd57 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:28.165576 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 23/43] 10.222.0.15 () {58 vars in 1361 bytes} [Mon Sep 5 14:13:27 2022] PUT /identity/v3/domains/default/users/40eedd635c074248bf2c6c67ebf16bb0/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:13:28.268644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:28.278658 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348d3aa-bf6b-4c9a-8374-288f7361cd57 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:28.316814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18c504-1147-478c-917d-b555ee1189d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:28.317369 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18c504-1147-478c-917d-b555ee1189d9 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:28.317662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18c504-1147-478c-917d-b555ee1189d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:28.317936 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18c504-1147-478c-917d-b555ee1189d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:28.318605 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18c504-1147-478c-917d-b555ee1189d9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:28.320121 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18c504-1147-478c-917d-b555ee1189d9 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:28.331235 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363dc34-b5ac-4f80-9033-65b25109491a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:28.399640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 21/44] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Sep 5 14:13:28 2022] POST /identity/v3/roles => generated 223 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:13:28.438704 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363dc34-b5ac-4f80-9033-65b25109491a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:28.476963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043d899-9659-4653-96b3-e4ead092a3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:28.477644 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043d899-9659-4653-96b3-e4ead092a3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:28.477929 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043d899-9659-4653-96b3-e4ead092a3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:28.478206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043d899-9659-4653-96b3-e4ead092a3c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:28.479987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043d899-9659-4653-96b3-e4ead092a3c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:28.481465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043d899-9659-4653-96b3-e4ead092a3c0 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:28.483278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 24/45] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:13:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:13:28.488631 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18c504-1147-478c-917d-b555ee1189d9 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:28.599060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18c504-1147-478c-917d-b555ee1189d9 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:28.633110 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab9308-a922-4efc-bbc8-db0fe0d534f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:28.633728 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab9308-a922-4efc-bbc8-db0fe0d534f0 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:28.634010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab9308-a922-4efc-bbc8-db0fe0d534f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:28.634282 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab9308-a922-4efc-bbc8-db0fe0d534f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:28.634974 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab9308-a922-4efc-bbc8-db0fe0d534f0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:28.636482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab9308-a922-4efc-bbc8-db0fe0d534f0 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:28.721168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 22/46] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 5 14:13:28 2022] POST /identity/v3/projects => generated 312 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:13:30.086915 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101290e7-1aeb-45a8-94e3-6f749e64eafb None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:30.087421 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101290e7-1aeb-45a8-94e3-6f749e64eafb None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:30.087505 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101290e7-1aeb-45a8-94e3-6f749e64eafb None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:30.088826 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 25/47] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:30.094666 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072036e-e034-4a46-a253-6d41e2cabf8b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:30.094666 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072036e-e034-4a46-a253-6d41e2cabf8b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:30.094666 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072036e-e034-4a46-a253-6d41e2cabf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:30.100973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:30.112798 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-f072036e-e034-4a46-a253-6d41e2cabf8b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:30.116083 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f072036e-e034-4a46-a253-6d41e2cabf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:30.178146 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 23/48] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:30.444652 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101290e7-1aeb-45a8-94e3-6f749e64eafb None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:30.466815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:30.545817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101290e7-1aeb-45a8-94e3-6f749e64eafb None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:30.578766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdafe5a-af75-4bc9-91e7-ee00fc7b3bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:30.579424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdafe5a-af75-4bc9-91e7-ee00fc7b3bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:30.579704 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdafe5a-af75-4bc9-91e7-ee00fc7b3bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:30.579976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdafe5a-af75-4bc9-91e7-ee00fc7b3bf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:30.590470 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdafe5a-af75-4bc9-91e7-ee00fc7b3bf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:30.595288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-2cdafe5a-af75-4bc9-91e7-ee00fc7b3bf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:13:30.595288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:30.595288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:30.595288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:30.595288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:30.595288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:30.595288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:30.595288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:30.595288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:30.595288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:30.595288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:30.595288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:30.595288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:30.595288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:30.595288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:30.595288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:30.596111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:30.596111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:30.596111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:30.596111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:30.596111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:30.596111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:30.596111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:30.596111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:30.596111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:30.596111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:30.596111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:30.596111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:30.596111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:30.596111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:30.596111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:30.596111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:30.596878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:30.596878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:30.596878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:30.596878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:30.596878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:30.596878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:30.596878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:30.596878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:30.596878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:30.596878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:30.596878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:13:30.596878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:13:30.596878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 26/49] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:13:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:30.603438 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072036e-e034-4a46-a253-6d41e2cabf8b None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:30.622981 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85ce40-b184-4712-bd72-6f563666c506 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:30.623274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85ce40-b184-4712-bd72-6f563666c506 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:30.623557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85ce40-b184-4712-bd72-6f563666c506 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:30.624883 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 27/50] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:13:30.629649 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf52e3-aaa9-47a7-8f7f-066af0090c7f None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:30.629932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf52e3-aaa9-47a7-8f7f-066af0090c7f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:30.630203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf52e3-aaa9-47a7-8f7f-066af0090c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:30.648333 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-7faf52e3-aaa9-47a7-8f7f-066af0090c7f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:30.649541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7faf52e3-aaa9-47a7-8f7f-066af0090c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:30.701137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072036e-e034-4a46-a253-6d41e2cabf8b None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:30.714677 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 28/51] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:13:30.729915 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:30.734603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f8edf-7388-4a64-8530-260ba3bdf9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:30.735245 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f8edf-7388-4a64-8530-260ba3bdf9b6 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:30.735530 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f8edf-7388-4a64-8530-260ba3bdf9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:30.735805 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f8edf-7388-4a64-8530-260ba3bdf9b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:30.736467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f8edf-7388-4a64-8530-260ba3bdf9b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:30.736770 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f8edf-7388-4a64-8530-260ba3bdf9b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:30.738155 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f8edf-7388-4a64-8530-260ba3bdf9b6 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:30.746120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 24/52] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:30.751614 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf52e3-aaa9-47a7-8f7f-066af0090c7f None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:30.763270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:30.847747 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf52e3-aaa9-47a7-8f7f-066af0090c7f None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:30.880523 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e881b1-6e9d-4732-9014-a755e6a90637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:30.881182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e881b1-6e9d-4732-9014-a755e6a90637 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:30.881494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e881b1-6e9d-4732-9014-a755e6a90637 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:30.881766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e881b1-6e9d-4732-9014-a755e6a90637 None admin] PATH_INFO: `/v3/users/40eedd635c074248bf2c6c67ebf16bb0` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:30.882911 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e881b1-6e9d-4732-9014-a755e6a90637 None admin] RBAC: Authorizing `identity:get_user(user_id=40eedd635c074248bf2c6c67ebf16bb0)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:30.884414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e881b1-6e9d-4732-9014-a755e6a90637 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:30.886584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 29/53] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:30 2022] GET /identity/v3/users/40eedd635c074248bf2c6c67ebf16bb0 => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:30.892310 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f8edf-7388-4a64-8530-260ba3bdf9b6 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:30.989485 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e881b1-6e9d-4732-9014-a755e6a90637 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:30.990337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f8edf-7388-4a64-8530-260ba3bdf9b6 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:31.025367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241dc29-fb1a-420c-8e6b-1cec2b971555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:31.025545 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241dc29-fb1a-420c-8e6b-1cec2b971555 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:31.026017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241dc29-fb1a-420c-8e6b-1cec2b971555 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:31.026299 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241dc29-fb1a-420c-8e6b-1cec2b971555 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:31.027430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241dc29-fb1a-420c-8e6b-1cec2b971555 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:31.028902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241dc29-fb1a-420c-8e6b-1cec2b971555 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:31.030653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 25/54] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:13:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:31.036239 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241dc29-fb1a-420c-8e6b-1cec2b971555 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:31.069850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:31.090936 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e881b1-6e9d-4732-9014-a755e6a90637 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:31.099940 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:31.145834 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241dc29-fb1a-420c-8e6b-1cec2b971555 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:31.181246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272013b5-90d2-46dd-9ca7-9dd9097c32dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:31.181878 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272013b5-90d2-46dd-9ca7-9dd9097c32dc None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:31.182950 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272013b5-90d2-46dd-9ca7-9dd9097c32dc None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:31.183237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272013b5-90d2-46dd-9ca7-9dd9097c32dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:31.183978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272013b5-90d2-46dd-9ca7-9dd9097c32dc None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.domain.id=default, user.id=40eedd635c074248bf2c6c67ebf16bb0) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:31.184289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272013b5-90d2-46dd-9ca7-9dd9097c32dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:31.185781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272013b5-90d2-46dd-9ca7-9dd9097c32dc None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:31.192636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 26/55] 10.222.0.15 () {56 vars in 1322 bytes} [Mon Sep 5 14:13:31 2022] GET /identity/v3/role_assignments?user.id=40eedd635c074248bf2c6c67ebf16bb0&scope.domain.id=default&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 530 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:13:31.232639 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1e511-4ec7-46a6-9e8b-ee45af3224fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:31.233292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1e511-4ec7-46a6-9e8b-ee45af3224fd None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:31.233578 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1e511-4ec7-46a6-9e8b-ee45af3224fd None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:31.233856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1e511-4ec7-46a6-9e8b-ee45af3224fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:31.235079 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1e511-4ec7-46a6-9e8b-ee45af3224fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:31.236590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1e511-4ec7-46a6-9e8b-ee45af3224fd None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:31.238294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 30/56] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:13:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:13:31.244304 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272013b5-90d2-46dd-9ca7-9dd9097c32dc None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:31.348185 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272013b5-90d2-46dd-9ca7-9dd9097c32dc None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:31.374302 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:31.382728 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb7e52-5d83-45ba-a411-7d266373d639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:31.383377 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb7e52-5d83-45ba-a411-7d266373d639 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:31.383664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb7e52-5d83-45ba-a411-7d266373d639 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:31.383948 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb7e52-5d83-45ba-a411-7d266373d639 None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:31.385145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb7e52-5d83-45ba-a411-7d266373d639 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:31.386131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb7e52-5d83-45ba-a411-7d266373d639 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:31.480456 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 27/57] 10.222.0.15 () {60 vars in 1140 bytes} [Mon Sep 5 14:13:31 2022] POST /identity/v3/projects => generated 298 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:13:33.372112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb852d-eaf6-471c-98cd-45bb317f01aa None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:33.372611 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb852d-eaf6-471c-98cd-45bb317f01aa None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:33.372611 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb852d-eaf6-471c-98cd-45bb317f01aa None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:33.373903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 31/58] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:33.379058 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a7376-b766-41f1-98ba-b23166c5e345 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:33.379422 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a7376-b766-41f1-98ba-b23166c5e345 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:33.379625 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a7376-b766-41f1-98ba-b23166c5e345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:33.397740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-215a7376-b766-41f1-98ba-b23166c5e345 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:33.398981 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-215a7376-b766-41f1-98ba-b23166c5e345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:33.462774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 28/59] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:33.729641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb852d-eaf6-471c-98cd-45bb317f01aa None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:33.827955 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb852d-eaf6-471c-98cd-45bb317f01aa None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:33.859510 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18437c91-5549-494c-9dab-2c4c2109f0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:33.860127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18437c91-5549-494c-9dab-2c4c2109f0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:33.860410 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18437c91-5549-494c-9dab-2c4c2109f0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:33.860724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18437c91-5549-494c-9dab-2c4c2109f0c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:33.861868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18437c91-5549-494c-9dab-2c4c2109f0c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:33.863328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18437c91-5549-494c-9dab-2c4c2109f0c7 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:33.864840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 32/60] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:13:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:33.870708 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a7376-b766-41f1-98ba-b23166c5e345 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:33.966540 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a7376-b766-41f1-98ba-b23166c5e345 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:33.975408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:33.999306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663ad09-98fd-4e0b-b816-958476f742e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:33.999847 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663ad09-98fd-4e0b-b816-958476f742e5 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:34.000129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663ad09-98fd-4e0b-b816-958476f742e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:34.000404 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663ad09-98fd-4e0b-b816-958476f742e5 None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:34.001058 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663ad09-98fd-4e0b-b816-958476f742e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:34.002491 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663ad09-98fd-4e0b-b816-958476f742e5 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:34.118627 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 29/61] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:13:33 2022] POST /identity/v3/users => generated 275 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:13:36.968850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e9544-26a3-4de1-8795-6a1ce30d1b14 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:36.969947 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e9544-26a3-4de1-8795-6a1ce30d1b14 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:36.970442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e9544-26a3-4de1-8795-6a1ce30d1b14 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:36.972024 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 33/62] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:36.977396 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3315c-14ca-420f-b992-d6c39013d0c9 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:36.977809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3315c-14ca-420f-b992-d6c39013d0c9 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:36.977983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3315c-14ca-420f-b992-d6c39013d0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:37.004171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-d0b3315c-14ca-420f-b992-d6c39013d0c9 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:37.005390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d0b3315c-14ca-420f-b992-d6c39013d0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:37.106290 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 30/63] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:37.306434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b27c9-e004-496e-9485-f4f2f3b39d5e None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:37.307709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b27c9-e004-496e-9485-f4f2f3b39d5e None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:37.308206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b27c9-e004-496e-9485-f4f2f3b39d5e None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:37.309761 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 34/64] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:37.315344 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4e88c-b293-40b6-857e-ce432ff67f36 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:37.315899 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4e88c-b293-40b6-857e-ce432ff67f36 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:37.316357 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4e88c-b293-40b6-857e-ce432ff67f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:37.343786 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-63c4e88c-b293-40b6-857e-ce432ff67f36 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:37.345271 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-63c4e88c-b293-40b6-857e-ce432ff67f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:37.435770 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 31/65] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:37.448197 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b27c9-e004-496e-9485-f4f2f3b39d5e None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:37.451283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb18703-f747-42f5-bc3d-6d56101c0673 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:37.451760 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb18703-f747-42f5-bc3d-6d56101c0673 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:37.452184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb18703-f747-42f5-bc3d-6d56101c0673 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:37.453681 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 32/66] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:13:37.458707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76f556-0076-41aa-8b64-c7b57b72a7b5 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:37.459172 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76f556-0076-41aa-8b64-c7b57b72a7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:37.459595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76f556-0076-41aa-8b64-c7b57b72a7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:37.480044 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:37.495829 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-ce76f556-0076-41aa-8b64-c7b57b72a7b5 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:37.497052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ce76f556-0076-41aa-8b64-c7b57b72a7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:37.529875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:37.622079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 33/67] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:37.631208 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b27c9-e004-496e-9485-f4f2f3b39d5e None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:37.676084 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7f3a2-0a2d-4ceb-8755-d9c64d7e4191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:37.676958 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7f3a2-0a2d-4ceb-8755-d9c64d7e4191 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:37.677447 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7f3a2-0a2d-4ceb-8755-d9c64d7e4191 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:37.677862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7f3a2-0a2d-4ceb-8755-d9c64d7e4191 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:37.689172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7f3a2-0a2d-4ceb-8755-d9c64d7e4191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:37.690221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e99085-f57d-42b0-8046-1b2d1a03d568 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:37.690560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e99085-f57d-42b0-8046-1b2d1a03d568 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:37.691100 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e99085-f57d-42b0-8046-1b2d1a03d568 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:37.692633 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 34/68] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:13:37.694723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-d5e7f3a2-0a2d-4ceb-8755-d9c64d7e4191 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:13:37.694723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:37.694723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:37.694723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:37.694723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:37.694723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:37.694723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:37.694723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:37.694723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:37.694723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:37.694723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:37.694723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:37.694723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:37.694723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:37.694723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:37.694723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:37.695572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:37.695572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:37.695572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:37.695572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:37.695572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:37.695572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:37.695572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:37.695572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:37.695572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:37.695572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:37.695572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:37.695572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:37.695572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:37.695572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:37.695572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:37.695572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:37.696335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:37.696335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:37.696335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:37.696335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:37.696335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:37.696335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:37.696335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:37.696335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:37.696335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:37.696335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:37.696335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:13:37.696335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:13:37.696335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 35/69] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:37.704146 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be4d44-cf09-4f02-8df4-8e51da1fed8b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:37.704146 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be4d44-cf09-4f02-8df4-8e51da1fed8b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:37.704146 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be4d44-cf09-4f02-8df4-8e51da1fed8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:37.706647 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7f3a2-0a2d-4ceb-8755-d9c64d7e4191 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:37.734542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-34be4d44-cf09-4f02-8df4-8e51da1fed8b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:37.736075 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-34be4d44-cf09-4f02-8df4-8e51da1fed8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:37.802648 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:37.813950 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 35/70] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:13:37.819868 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be4d44-cf09-4f02-8df4-8e51da1fed8b None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:37.841740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7f3a2-0a2d-4ceb-8755-d9c64d7e4191 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:37.889962 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd3131-0eb9-4315-aac3-a3814af3fcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:37.890837 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd3131-0eb9-4315-aac3-a3814af3fcbf None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:37.891265 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd3131-0eb9-4315-aac3-a3814af3fcbf None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:37.891699 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd3131-0eb9-4315-aac3-a3814af3fcbf None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:37.892506 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd3131-0eb9-4315-aac3-a3814af3fcbf None admin] RBAC: Adding query filter params (name=member) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:37.892954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd3131-0eb9-4315-aac3-a3814af3fcbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:37.894519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd3131-0eb9-4315-aac3-a3814af3fcbf None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:37.907194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 36/71] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:13:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:37.908604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced313db-ddd6-4bd4-8524-3b21a610ce92 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:37.909033 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced313db-ddd6-4bd4-8524-3b21a610ce92 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:37.909461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced313db-ddd6-4bd4-8524-3b21a610ce92 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:37.910823 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 37/72] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:37.913812 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced313db-ddd6-4bd4-8524-3b21a610ce92 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:37.941315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:37.957904 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be4d44-cf09-4f02-8df4-8e51da1fed8b None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:37.993000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abf766-e5b2-429e-aa4a-f385f7f055d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:37.993643 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abf766-e5b2-429e-aa4a-f385f7f055d9 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:37.993932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abf766-e5b2-429e-aa4a-f385f7f055d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:37.994212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abf766-e5b2-429e-aa4a-f385f7f055d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:38.005855 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3abf766-e5b2-429e-aa4a-f385f7f055d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:38.011952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-c3abf766-e5b2-429e-aa4a-f385f7f055d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:13:38.011952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:38.011952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:38.011952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:38.011952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:38.011952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:38.011952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:38.011952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:38.011952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:38.011952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:38.011952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:38.011952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:38.011952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:38.011952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:38.011952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:38.011952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:38.012797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:38.012797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:38.012797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:38.012797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:38.012797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:38.012797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:38.012797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:38.012797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:38.012797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:38.012797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:38.012797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:38.012797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:38.012797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:38.012797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:38.012797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:38.012797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:38.013556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:38.013556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:38.013556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:38.013556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:38.013556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:38.013556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:38.013556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:38.013556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:38.013556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:38.013556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:38.013556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:13:38.013556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:13:38.013556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 36/73] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:13:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:38.014662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b3b07-5413-478c-a214-d8bc229cd477 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:38.014662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b3b07-5413-478c-a214-d8bc229cd477 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:38.014957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b3b07-5413-478c-a214-d8bc229cd477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:38.032773 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-423b3b07-5413-478c-a214-d8bc229cd477 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:38.033964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-423b3b07-5413-478c-a214-d8bc229cd477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:38.035872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced313db-ddd6-4bd4-8524-3b21a610ce92 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:38.072918 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11967a-019d-4324-807d-31f4604a9814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:38.073747 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11967a-019d-4324-807d-31f4604a9814 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:38.074176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11967a-019d-4324-807d-31f4604a9814 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:38.074574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11967a-019d-4324-807d-31f4604a9814 None admin] PATH_INFO: `/v3/users/b0cca89fab4740169878ca61aabace33` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:38.080592 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:38.093627 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11967a-019d-4324-807d-31f4604a9814 None admin] RBAC: Authorizing `identity:get_user(user_id=b0cca89fab4740169878ca61aabace33)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:38.095551 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11967a-019d-4324-807d-31f4604a9814 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:38.097395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 38/74] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:37 2022] GET /identity/v3/users/b0cca89fab4740169878ca61aabace33 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:38.097997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11967a-019d-4324-807d-31f4604a9814 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:38.122962 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 37/75] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:38.123814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b3b07-5413-478c-a214-d8bc229cd477 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:38.199924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:38.207937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11967a-019d-4324-807d-31f4604a9814 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:38.224260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b3b07-5413-478c-a214-d8bc229cd477 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:38.244791 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f71f3-9df8-4f52-ad8b-c66e3a883477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:38.245440 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f71f3-9df8-4f52-ad8b-c66e3a883477 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:38.245726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f71f3-9df8-4f52-ad8b-c66e3a883477 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:38.246007 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f71f3-9df8-4f52-ad8b-c66e3a883477 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:38.257384 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f71f3-9df8-4f52-ad8b-c66e3a883477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:38.260945 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab04ef3-1110-4ae0-bc7d-cd1c82a76e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:38.261560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab04ef3-1110-4ae0-bc7d-cd1c82a76e48 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:38.261848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab04ef3-1110-4ae0-bc7d-cd1c82a76e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:38.262128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab04ef3-1110-4ae0-bc7d-cd1c82a76e48 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:38.262223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-a37f71f3-9df8-4f52-ad8b-c66e3a883477 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:13:38.262223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:38.262223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:38.262223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:38.262223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:38.262223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:38.262223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:38.262223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:38.262223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:38.262223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:38.262223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:38.262223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:38.262223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:38.262223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:38.262223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:38.262223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:38.263061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:38.263061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:38.263061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:38.263061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:38.263061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:38.263061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:38.263061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:38.263061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:38.263061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:38.263061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:38.263061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:38.263061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:38.263061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:38.263061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:38.263061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:38.263061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:38.263930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:38.263930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:38.263930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:38.263930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:38.263930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:38.263930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:38.263930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:38.263930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:38.263930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:38.263930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:38.263930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:13:38.263930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:13:38.264516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab04ef3-1110-4ae0-bc7d-cd1c82a76e48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:38.264516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab04ef3-1110-4ae0-bc7d-cd1c82a76e48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:38.264642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 39/76] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:13:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:38.265640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f71f3-9df8-4f52-ad8b-c66e3a883477 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:38.269135 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab04ef3-1110-4ae0-bc7d-cd1c82a76e48 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:38.283216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 38/77] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:38.283216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab04ef3-1110-4ae0-bc7d-cd1c82a76e48 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:38.328436 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:38.374873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f71f3-9df8-4f52-ad8b-c66e3a883477 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:38.386587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab04ef3-1110-4ae0-bc7d-cd1c82a76e48 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:38.409130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e73727-19aa-4bba-9232-7a928b02dc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:38.409748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e73727-19aa-4bba-9232-7a928b02dc5a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:38.410033 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e73727-19aa-4bba-9232-7a928b02dc5a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:38.410324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e73727-19aa-4bba-9232-7a928b02dc5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:38.420552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be840c-617e-4fbf-8ea4-593cccf8cc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:38.421189 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be840c-617e-4fbf-8ea4-593cccf8cc94 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:38.421472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be840c-617e-4fbf-8ea4-593cccf8cc94 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:38.421752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be840c-617e-4fbf-8ea4-593cccf8cc94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:38.422904 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be840c-617e-4fbf-8ea4-593cccf8cc94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:38.424373 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be840c-617e-4fbf-8ea4-593cccf8cc94 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:38.426009 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 39/78] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:13:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:38.426710 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be840c-617e-4fbf-8ea4-593cccf8cc94 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:38.433834 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:38.482810 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e73727-19aa-4bba-9232-7a928b02dc5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:38.484326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e73727-19aa-4bba-9232-7a928b02dc5a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:38.527437 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be840c-617e-4fbf-8ea4-593cccf8cc94 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:38.536310 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:38.560494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8b1ce-0721-4e59-adee-62b9e350177b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:38.561095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8b1ce-0721-4e59-adee-62b9e350177b None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:38.561379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8b1ce-0721-4e59-adee-62b9e350177b None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:38.561658 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8b1ce-0721-4e59-adee-62b9e350177b None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:38.562326 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b8b1ce-0721-4e59-adee-62b9e350177b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:38.562631 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b8b1ce-0721-4e59-adee-62b9e350177b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:38.564025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b8b1ce-0721-4e59-adee-62b9e350177b None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:38.572016 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 40/79] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:13:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:38.572869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8b1ce-0721-4e59-adee-62b9e350177b None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:38.586151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 40/80] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:13:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:13:38.587014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e73727-19aa-4bba-9232-7a928b02dc5a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:38.639353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:38.671988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8b1ce-0721-4e59-adee-62b9e350177b None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:38.684068 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e73727-19aa-4bba-9232-7a928b02dc5a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:38.705905 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125862b-0624-4436-a2a3-0f2b467252a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:38.706560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125862b-0624-4436-a2a3-0f2b467252a6 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:38.706877 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125862b-0624-4436-a2a3-0f2b467252a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:38.707160 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125862b-0624-4436-a2a3-0f2b467252a6 None admin] PATH_INFO: `/v3/users/40eedd635c074248bf2c6c67ebf16bb0` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:38.708279 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125862b-0624-4436-a2a3-0f2b467252a6 None admin] RBAC: Authorizing `identity:get_user(user_id=40eedd635c074248bf2c6c67ebf16bb0)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:38.709728 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125862b-0624-4436-a2a3-0f2b467252a6 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:38.711330 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 41/81] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:38 2022] GET /identity/v3/users/40eedd635c074248bf2c6c67ebf16bb0 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:38.712060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125862b-0624-4436-a2a3-0f2b467252a6 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:38.720372 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7aa6f-8581-4057-b521-23326ed6ca71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:38.721006 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7aa6f-8581-4057-b521-23326ed6ca71 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:38.721289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7aa6f-8581-4057-b521-23326ed6ca71 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:38.721565 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7aa6f-8581-4057-b521-23326ed6ca71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:38.731729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7aa6f-8581-4057-b521-23326ed6ca71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:38.736595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-bac7aa6f-8581-4057-b521-23326ed6ca71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:13:38.736595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:38.736595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:38.736595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:38.736595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:38.736595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:38.736595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:38.736595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:38.736595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:38.736595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:38.736595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:38.736595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:38.736595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:38.736595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:38.736595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:38.736595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:38.737542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:38.737542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:38.737542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:38.737542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:38.737542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:38.737542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:38.737542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:38.737542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:38.737542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:38.737542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:38.737542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:38.737542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:38.737542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:38.737542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:38.737542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:38.737542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:38.738315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:38.738315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:38.738315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:38.738315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:38.738315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:38.738315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:38.738315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:38.738315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:38.738315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:38.738315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:38.738315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:13:38.738315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:13:38.738315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 41/82] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:38.743143 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7aa6f-8581-4057-b521-23326ed6ca71 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:38.758937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:38.810876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125862b-0624-4436-a2a3-0f2b467252a6 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:38.841037 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7aa6f-8581-4057-b521-23326ed6ca71 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:38.844751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea475432-6e8d-4681-87d0-2ad786faa45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:38.845360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea475432-6e8d-4681-87d0-2ad786faa45d None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:38.845650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea475432-6e8d-4681-87d0-2ad786faa45d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:38.845926 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea475432-6e8d-4681-87d0-2ad786faa45d None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:38.846573 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea475432-6e8d-4681-87d0-2ad786faa45d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:38.848070 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea475432-6e8d-4681-87d0-2ad786faa45d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:38.856120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:38.874023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c481509-d7a6-483f-bfaf-4282e7bad2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:38.874658 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c481509-d7a6-483f-bfaf-4282e7bad2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:38.874979 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c481509-d7a6-483f-bfaf-4282e7bad2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:38.875257 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c481509-d7a6-483f-bfaf-4282e7bad2f1 None admin] PATH_INFO: `/v3/users/b0cca89fab4740169878ca61aabace33` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:38.876390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c481509-d7a6-483f-bfaf-4282e7bad2f1 None admin] RBAC: Authorizing `identity:get_user(user_id=b0cca89fab4740169878ca61aabace33)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:38.877835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c481509-d7a6-483f-bfaf-4282e7bad2f1 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:38.879434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 42/83] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:38 2022] GET /identity/v3/users/b0cca89fab4740169878ca61aabace33 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:38.880272 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c481509-d7a6-483f-bfaf-4282e7bad2f1 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:38.953280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 42/84] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:13:38 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:13:38.954185 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea475432-6e8d-4681-87d0-2ad786faa45d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:38.965148 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:38.985808 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c481509-d7a6-483f-bfaf-4282e7bad2f1 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:39.023677 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186ce09-a406-4779-93e1-b8baa51834eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:39.024302 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186ce09-a406-4779-93e1-b8baa51834eb None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:39.024593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186ce09-a406-4779-93e1-b8baa51834eb None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:39.024872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186ce09-a406-4779-93e1-b8baa51834eb None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:39.026119 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186ce09-a406-4779-93e1-b8baa51834eb None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:39.027707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186ce09-a406-4779-93e1-b8baa51834eb None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:39.030344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 43/85] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:38 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:39.031422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186ce09-a406-4779-93e1-b8baa51834eb None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:39.072724 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea475432-6e8d-4681-87d0-2ad786faa45d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:39.116577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee8572-f5a0-4220-9489-61cbf909c3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:39.117203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee8572-f5a0-4220-9489-61cbf909c3bd None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:39.117492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee8572-f5a0-4220-9489-61cbf909c3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:39.117775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee8572-f5a0-4220-9489-61cbf909c3bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:39.151985 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186ce09-a406-4779-93e1-b8baa51834eb None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:39.198221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:39.198577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcdd33-dbd5-4879-9e33-85e8a1b45c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:39.199245 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcdd33-dbd5-4879-9e33-85e8a1b45c8f None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:39.199527 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcdd33-dbd5-4879-9e33-85e8a1b45c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:39.199806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcdd33-dbd5-4879-9e33-85e8a1b45c8f None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:39.200574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcdd33-dbd5-4879-9e33-85e8a1b45c8f None admin] RBAC: Adding query filter params (name=member) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:39.200791 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcdd33-dbd5-4879-9e33-85e8a1b45c8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:39.202181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcdd33-dbd5-4879-9e33-85e8a1b45c8f None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:39.211291 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee8572-f5a0-4220-9489-61cbf909c3bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:39.212744 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee8572-f5a0-4220-9489-61cbf909c3bd None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:39.214987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 44/86] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:13:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:39.215845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcdd33-dbd5-4879-9e33-85e8a1b45c8f None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:39.240377 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:39.335654 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcdd33-dbd5-4879-9e33-85e8a1b45c8f None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:39.353151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 43/87] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:13:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:13:39.354570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee8572-f5a0-4220-9489-61cbf909c3bd None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:39.377493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e01862-6dde-4081-b9ec-611b7212aa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:39.378165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e01862-6dde-4081-b9ec-611b7212aa53 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:39.378454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e01862-6dde-4081-b9ec-611b7212aa53 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:39.378732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e01862-6dde-4081-b9ec-611b7212aa53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:39.459659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e01862-6dde-4081-b9ec-611b7212aa53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:39.461884 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e01862-6dde-4081-b9ec-611b7212aa53 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:39.471011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:39.482992 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:39.483680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee8572-f5a0-4220-9489-61cbf909c3bd None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:39.520838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a40a8-739e-4828-95d5-b881cc977497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:39.521453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a40a8-739e-4828-95d5-b881cc977497 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:39.521735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a40a8-739e-4828-95d5-b881cc977497 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:39.522007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a40a8-739e-4828-95d5-b881cc977497 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:39.522740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a40a8-739e-4828-95d5-b881cc977497 None admin] RBAC: Adding query filter params (role.id=5812e6538f3f4bdfbfbe11cd25c02957, scope.project.id=2269c4a7f991405497de468e3a9a745a, user.id=b0cca89fab4740169878ca61aabace33) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:39.523212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a40a8-739e-4828-95d5-b881cc977497 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:39.524731 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a40a8-739e-4828-95d5-b881cc977497 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:39.534926 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 44/88] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:13:39 2022] GET /identity/v3/role_assignments?user.id=b0cca89fab4740169878ca61aabace33&scope.project.id=2269c4a7f991405497de468e3a9a745a&role.id=5812e6538f3f4bdfbfbe11cd25c02957 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:39.535943 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a40a8-739e-4828-95d5-b881cc977497 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:39.613055 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 45/89] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:13:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:13:39.613863 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e01862-6dde-4081-b9ec-611b7212aa53 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:39.745394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a40a8-739e-4828-95d5-b881cc977497 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:39.777958 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e01862-6dde-4081-b9ec-611b7212aa53 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:39.794316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df9ab0-bfa8-4272-ac2e-10f259c949fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:39.794993 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df9ab0-bfa8-4272-ac2e-10f259c949fb None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:39.795287 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df9ab0-bfa8-4272-ac2e-10f259c949fb None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:39.795573 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df9ab0-bfa8-4272-ac2e-10f259c949fb None admin] PATH_INFO: `/v3/users/b0cca89fab4740169878ca61aabace33` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:39.798383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df9ab0-bfa8-4272-ac2e-10f259c949fb None admin] RBAC: Authorizing `identity:get_user(user_id=b0cca89fab4740169878ca61aabace33)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:39.799940 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df9ab0-bfa8-4272-ac2e-10f259c949fb None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:39.801554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 45/90] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:39 2022] GET /identity/v3/users/b0cca89fab4740169878ca61aabace33 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:39.802329 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df9ab0-bfa8-4272-ac2e-10f259c949fb None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:39.807708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:39.846578 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ea3e0-8cde-48dd-8db7-2d6ada9b89b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:39.847251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ea3e0-8cde-48dd-8db7-2d6ada9b89b4 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:39.847537 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ea3e0-8cde-48dd-8db7-2d6ada9b89b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:39.847817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ea3e0-8cde-48dd-8db7-2d6ada9b89b4 None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:39.849127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ea3e0-8cde-48dd-8db7-2d6ada9b89b4 None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:39.853466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ea3e0-8cde-48dd-8db7-2d6ada9b89b4 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:39.855589 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 46/91] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:39 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:39.856430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ea3e0-8cde-48dd-8db7-2d6ada9b89b4 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:39.862026 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:39.967016 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ea3e0-8cde-48dd-8db7-2d6ada9b89b4 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:39.998358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df9ab0-bfa8-4272-ac2e-10f259c949fb None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:40.008534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d2cb2-b756-4935-b0b0-83f97773b0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:40.008534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d2cb2-b756-4935-b0b0-83f97773b0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:40.008534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d2cb2-b756-4935-b0b0-83f97773b0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:40.008534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d2cb2-b756-4935-b0b0-83f97773b0e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:40.065574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:40.067697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d99dd9-5f50-4963-8ac4-fcd30f7f416e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:40.068605 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d99dd9-5f50-4963-8ac4-fcd30f7f416e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:40.069105 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d99dd9-5f50-4963-8ac4-fcd30f7f416e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:40.069593 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d99dd9-5f50-4963-8ac4-fcd30f7f416e None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:40.075574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d99dd9-5f50-4963-8ac4-fcd30f7f416e None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:40.077732 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d99dd9-5f50-4963-8ac4-fcd30f7f416e None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:40.079986 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 46/92] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:39 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:40.091857 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d99dd9-5f50-4963-8ac4-fcd30f7f416e None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:40.091986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d2cb2-b756-4935-b0b0-83f97773b0e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:40.091986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d2cb2-b756-4935-b0b0-83f97773b0e5 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:40.195771 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 47/93] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:13:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:13:40.197291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d2cb2-b756-4935-b0b0-83f97773b0e5 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:40.274107 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:40.289424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:40.296946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d99dd9-5f50-4963-8ac4-fcd30f7f416e None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:40.313421 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d2cb2-b756-4935-b0b0-83f97773b0e5 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:40.352366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64847d5b-5a4e-4e05-80ae-2361009cb599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:40.353316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64847d5b-5a4e-4e05-80ae-2361009cb599 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:40.354269 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64847d5b-5a4e-4e05-80ae-2361009cb599 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:40.354691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64847d5b-5a4e-4e05-80ae-2361009cb599 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:40.355586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64847d5b-5a4e-4e05-80ae-2361009cb599 None admin] RBAC: Adding query filter params (role.id=1182ffc3e9f9477db2003500b7bfd402, scope.project.id=2269c4a7f991405497de468e3a9a745a, user.id=b0cca89fab4740169878ca61aabace33) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:40.356059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64847d5b-5a4e-4e05-80ae-2361009cb599 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:40.357707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64847d5b-5a4e-4e05-80ae-2361009cb599 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:40.365937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 48/94] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:13:40 2022] GET /identity/v3/role_assignments?user.id=b0cca89fab4740169878ca61aabace33&scope.project.id=2269c4a7f991405497de468e3a9a745a&role.id=1182ffc3e9f9477db2003500b7bfd402 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:40.366973 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64847d5b-5a4e-4e05-80ae-2361009cb599 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:40.380814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502bd03-41d1-42af-957f-01485005b17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:40.381462 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502bd03-41d1-42af-957f-01485005b17d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:40.381751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502bd03-41d1-42af-957f-01485005b17d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:40.382033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502bd03-41d1-42af-957f-01485005b17d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:40.382914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502bd03-41d1-42af-957f-01485005b17d None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=2269c4a7f991405497de468e3a9a745a, user.id=40eedd635c074248bf2c6c67ebf16bb0) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:40.383231 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502bd03-41d1-42af-957f-01485005b17d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:40.384723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502bd03-41d1-42af-957f-01485005b17d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:40.399932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 47/95] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:13:40 2022] GET /identity/v3/role_assignments?user.id=40eedd635c074248bf2c6c67ebf16bb0&scope.project.id=2269c4a7f991405497de468e3a9a745a&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 259 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:40.502442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64847d5b-5a4e-4e05-80ae-2361009cb599 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:40.540294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6b7c6-f115-430a-b28d-d609bd3d48c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:40.541237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6b7c6-f115-430a-b28d-d609bd3d48c9 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:40.541670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6b7c6-f115-430a-b28d-d609bd3d48c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:40.542114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6b7c6-f115-430a-b28d-d609bd3d48c9 None admin] PATH_INFO: `/v3/projects/4a0c087d2d50474e856da494b5d6563b` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:40.543724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c6b7c6-f115-430a-b28d-d609bd3d48c9 None admin] RBAC: Authorizing `identity:get_project(project_id=4a0c087d2d50474e856da494b5d6563b)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:40.545427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c6b7c6-f115-430a-b28d-d609bd3d48c9 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:40.547289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 49/96] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:40 2022] GET /identity/v3/projects/4a0c087d2d50474e856da494b5d6563b => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:40.555508 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502bd03-41d1-42af-957f-01485005b17d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:40.687466 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502bd03-41d1-42af-957f-01485005b17d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:40.732575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebd44e-2140-4f73-9aed-27a0ccd58644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:40.733453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebd44e-2140-4f73-9aed-27a0ccd58644 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:40.733888 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebd44e-2140-4f73-9aed-27a0ccd58644 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:40.734310 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebd44e-2140-4f73-9aed-27a0ccd58644 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:40.735247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ebd44e-2140-4f73-9aed-27a0ccd58644 None admin] RBAC: Adding query filter params (role.id=5812e6538f3f4bdfbfbe11cd25c02957, scope.project.id=4a0c087d2d50474e856da494b5d6563b, user.id=b0cca89fab4740169878ca61aabace33) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:40.735773 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ebd44e-2140-4f73-9aed-27a0ccd58644 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:40.737373 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ebd44e-2140-4f73-9aed-27a0ccd58644 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:40.748359 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 48/97] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:13:40 2022] GET /identity/v3/role_assignments?user.id=b0cca89fab4740169878ca61aabace33&scope.project.id=4a0c087d2d50474e856da494b5d6563b&role.id=5812e6538f3f4bdfbfbe11cd25c02957 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:41.364694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5642e4-f89d-4eee-9198-9d0b5fceefb2 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:41.365198 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5642e4-f89d-4eee-9198-9d0b5fceefb2 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:41.365282 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5642e4-f89d-4eee-9198-9d0b5fceefb2 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:41.366963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 50/98] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:41.372096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebf39b-c76d-450d-9a13-14f18132208e None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:41.372575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebf39b-c76d-450d-9a13-14f18132208e None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:41.372663 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebf39b-c76d-450d-9a13-14f18132208e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:41.395487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-e9ebf39b-c76d-450d-9a13-14f18132208e None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:41.396761 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e9ebf39b-c76d-450d-9a13-14f18132208e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:41.411251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:41.479843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 49/99] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:41.695632 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae3272-c6f9-45eb-b27e-c18e8d1d2225 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:41.695960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae3272-c6f9-45eb-b27e-c18e8d1d2225 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:41.696177 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae3272-c6f9-45eb-b27e-c18e8d1d2225 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:41.697673 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 51/100] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:41.702927 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74089d34-7a27-4808-9fe9-b3857d4c3bb1 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:41.703223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74089d34-7a27-4808-9fe9-b3857d4c3bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:41.703503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74089d34-7a27-4808-9fe9-b3857d4c3bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:41.727048 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-74089d34-7a27-4808-9fe9-b3857d4c3bb1 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:41.728258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-74089d34-7a27-4808-9fe9-b3857d4c3bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:41.819168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 50/101] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:42.024676 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae3272-c6f9-45eb-b27e-c18e8d1d2225 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:42.041176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:42.135963 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe3724-d77d-4270-86ed-3f0ba01e5f95 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:42.136283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe3724-d77d-4270-86ed-3f0ba01e5f95 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:42.136540 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe3724-d77d-4270-86ed-3f0ba01e5f95 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:42.137897 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 51/102] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:13:42.142727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e4fc8-f7c7-4194-9218-75d6ce9fdd74 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:42.143077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e4fc8-f7c7-4194-9218-75d6ce9fdd74 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:42.143327 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e4fc8-f7c7-4194-9218-75d6ce9fdd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:42.175380 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-8d9e4fc8-f7c7-4194-9218-75d6ce9fdd74 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:42.176586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8d9e4fc8-f7c7-4194-9218-75d6ce9fdd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:42.224982 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae3272-c6f9-45eb-b27e-c18e8d1d2225 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:42.302581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39449d-28fe-4dc4-b0fb-8452c0f1e416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:42.303178 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39449d-28fe-4dc4-b0fb-8452c0f1e416 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:42.303464 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39449d-28fe-4dc4-b0fb-8452c0f1e416 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:42.303740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39449d-28fe-4dc4-b0fb-8452c0f1e416 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:42.305889 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39449d-28fe-4dc4-b0fb-8452c0f1e416 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:42.307941 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39449d-28fe-4dc4-b0fb-8452c0f1e416 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:42.309531 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 52/103] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:13:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:42.315666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39449d-28fe-4dc4-b0fb-8452c0f1e416 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:42.339975 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 52/104] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:13:42.340968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e4fc8-f7c7-4194-9218-75d6ce9fdd74 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:42.363164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:42.531184 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:42.538623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39449d-28fe-4dc4-b0fb-8452c0f1e416 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:42.578158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e4fc8-f7c7-4194-9218-75d6ce9fdd74 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:42.603533 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169804f0-17db-4d56-a401-d8982a712239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:42.604561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169804f0-17db-4d56-a401-d8982a712239 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:42.605067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169804f0-17db-4d56-a401-d8982a712239 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:42.605562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169804f0-17db-4d56-a401-d8982a712239 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:42.624250 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169804f0-17db-4d56-a401-d8982a712239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:42.641156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-169804f0-17db-4d56-a401-d8982a712239 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:13:42.641156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:42.641156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:42.641156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:42.641156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:42.641156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:42.641156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:42.641156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:42.641156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:42.641156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:42.641156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:42.641156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:42.641156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:42.641156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:42.641156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:42.641156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:42.642455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:42.642455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:42.642455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:42.642455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:42.642455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:42.642455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:42.642455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:42.642455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:42.642455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:42.642455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:42.642455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:42.642455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:42.642455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:42.642455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:42.642455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:42.642455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:42.643655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:42.643655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:42.643655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:42.643655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:42.643655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:42.643655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:42.643655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:42.643655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:42.643655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:42.643655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:42.643655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:13:42.643655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:13:42.648420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bab89-20c5-4747-833e-533655245280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:42.649331 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bab89-20c5-4747-833e-533655245280 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:42.649821 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bab89-20c5-4747-833e-533655245280 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:42.650300 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bab89-20c5-4747-833e-533655245280 None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:42.651213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bab89-20c5-4747-833e-533655245280 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:42.652896 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bab89-20c5-4747-833e-533655245280 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:42.653017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 53/105] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:42.654430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169804f0-17db-4d56-a401-d8982a712239 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:42.761260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 53/106] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 5 14:13:42 2022] POST /identity/v3/projects => generated 302 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:13:42.842252 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169804f0-17db-4d56-a401-d8982a712239 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:42.888418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7f77c-efd7-4789-a9fc-273473477906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:42.889093 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7f77c-efd7-4789-a9fc-273473477906 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:42.889382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7f77c-efd7-4789-a9fc-273473477906 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:42.889710 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7f77c-efd7-4789-a9fc-273473477906 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:42.890399 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7f77c-efd7-4789-a9fc-273473477906 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:42.890708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7f77c-efd7-4789-a9fc-273473477906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:42.892156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7f77c-efd7-4789-a9fc-273473477906 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:42.907263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 54/107] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:13:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:42.911939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bab89-20c5-4747-833e-533655245280 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:42.998876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:43.019878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7f77c-efd7-4789-a9fc-273473477906 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:43.034830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:43.050217 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bab89-20c5-4747-833e-533655245280 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:43.097352 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d1654-5409-4ef5-ade1-611ddb33b840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:43.098064 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d1654-5409-4ef5-ade1-611ddb33b840 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:43.098357 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d1654-5409-4ef5-ade1-611ddb33b840 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:43.098638 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d1654-5409-4ef5-ade1-611ddb33b840 None admin] PATH_INFO: `/v3/users/0bb1f8cff11d4177879e801bed0e8db4` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:43.116291 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d1654-5409-4ef5-ade1-611ddb33b840 None admin] RBAC: Authorizing `identity:get_user(user_id=0bb1f8cff11d4177879e801bed0e8db4)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:43.118148 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d1654-5409-4ef5-ade1-611ddb33b840 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:43.120297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 54/108] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:42 2022] GET /identity/v3/users/0bb1f8cff11d4177879e801bed0e8db4 => generated 289 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:43.122122 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdbca1-5a61-4201-aa00-dc393c4e0c25 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:43.122422 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdbca1-5a61-4201-aa00-dc393c4e0c25 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:43.122704 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdbca1-5a61-4201-aa00-dc393c4e0c25 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:43.124427 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 55/109] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:43.128835 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fdbca1-5a61-4201-aa00-dc393c4e0c25 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:43.169865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7f77c-efd7-4789-a9fc-273473477906 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:43.209136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22c324-79b7-4007-bba7-8ff3e493c5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:43.210139 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22c324-79b7-4007-bba7-8ff3e493c5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:43.210648 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22c324-79b7-4007-bba7-8ff3e493c5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:43.211116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22c324-79b7-4007-bba7-8ff3e493c5d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:43.222196 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22c324-79b7-4007-bba7-8ff3e493c5d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:43.227479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-7d22c324-79b7-4007-bba7-8ff3e493c5d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:13:43.227479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:43.227479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:43.227479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:43.227479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:43.227479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:43.227479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:43.227479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:43.227479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:43.227479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:43.227479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:43.227479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:43.227479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:43.227479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:43.227479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:43.227479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:43.228349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:43.228349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:43.228349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:43.228349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:43.228349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:43.228349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:43.228349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:43.228349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:43.228349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:43.228349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:43.228349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:43.228349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:43.228349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:43.228349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:43.228349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:43.228349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:43.229661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:43.229661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:43.229661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:43.229661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:43.229661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:43.229661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:43.229661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:43.229661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:43.229661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:43.229661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:43.229661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:13:43.229661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:13:43.231665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 55/110] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:43.233529 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837cb0e-cc8c-418a-b1d6-e368a796d280 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:43.234015 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837cb0e-cc8c-418a-b1d6-e368a796d280 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:43.234469 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837cb0e-cc8c-418a-b1d6-e368a796d280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:43.243896 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fdbca1-5a61-4201-aa00-dc393c4e0c25 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:43.253264 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-e837cb0e-cc8c-418a-b1d6-e368a796d280 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:43.254845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e837cb0e-cc8c-418a-b1d6-e368a796d280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:43.264717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:43.284801 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c177f-9232-4635-9097-ac0d9a25dd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:43.285490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c177f-9232-4635-9097-ac0d9a25dd1b None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:43.285850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c177f-9232-4635-9097-ac0d9a25dd1b None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:43.286201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c177f-9232-4635-9097-ac0d9a25dd1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:43.339046 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:43.361647 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 56/111] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:43.361647 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837cb0e-cc8c-418a-b1d6-e368a796d280 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:43.379172 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c177f-9232-4635-9097-ac0d9a25dd1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:43.381024 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c177f-9232-4635-9097-ac0d9a25dd1b None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:43.478956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837cb0e-cc8c-418a-b1d6-e368a796d280 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:43.564707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995e3c7-6ffa-4622-b7df-b93fab5e357e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:43.565352 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995e3c7-6ffa-4622-b7df-b93fab5e357e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:43.565609 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995e3c7-6ffa-4622-b7df-b93fab5e357e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:43.565888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995e3c7-6ffa-4622-b7df-b93fab5e357e None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:43.566564 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995e3c7-6ffa-4622-b7df-b93fab5e357e None admin] RBAC: Adding query filter params (name=member) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:43.566911 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995e3c7-6ffa-4622-b7df-b93fab5e357e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:43.568308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995e3c7-6ffa-4622-b7df-b93fab5e357e None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:43.572420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 56/112] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:13:43.573840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb58f0-b02e-441e-bff0-547123522016 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:43.574143 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb58f0-b02e-441e-bff0-547123522016 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:43.574424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb58f0-b02e-441e-bff0-547123522016 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:43.575840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 57/113] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:43.583591 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18ac99-1cc7-4146-952f-79bd9ee04847 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:43.584129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18ac99-1cc7-4146-952f-79bd9ee04847 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:43.584129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18ac99-1cc7-4146-952f-79bd9ee04847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:43.589386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 57/114] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:13:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:43.590315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995e3c7-6ffa-4622-b7df-b93fab5e357e None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:43.629917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-5c18ac99-1cc7-4146-952f-79bd9ee04847 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:43.632991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-5c18ac99-1cc7-4146-952f-79bd9ee04847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:43.660335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:43.709564 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:43.738306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 58/115] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:43.743671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18ac99-1cc7-4146-952f-79bd9ee04847 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:43.771030 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995e3c7-6ffa-4622-b7df-b93fab5e357e None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:43.818366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa44160-b017-4d70-8222-b73036bdd52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:43.819333 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa44160-b017-4d70-8222-b73036bdd52c None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:43.819811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa44160-b017-4d70-8222-b73036bdd52c None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:43.820256 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa44160-b017-4d70-8222-b73036bdd52c None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:43.822996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa44160-b017-4d70-8222-b73036bdd52c None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:43.824917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa44160-b017-4d70-8222-b73036bdd52c None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:43.832733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 58/116] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:43 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:43.834694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0a435-9335-4443-84d9-09ac22ebc3e9 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:43.835181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0a435-9335-4443-84d9-09ac22ebc3e9 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:43.835650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0a435-9335-4443-84d9-09ac22ebc3e9 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:43.837546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 59/117] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:43.839430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0a435-9335-4443-84d9-09ac22ebc3e9 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:43.866142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18ac99-1cc7-4146-952f-79bd9ee04847 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:43.907278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fd8ab-9ff3-40d2-9e19-1642729aadfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:43.908234 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fd8ab-9ff3-40d2-9e19-1642729aadfe None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:43.908503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fd8ab-9ff3-40d2-9e19-1642729aadfe None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:43.908855 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fd8ab-9ff3-40d2-9e19-1642729aadfe None admin] PATH_INFO: `/v3/users/b0cca89fab4740169878ca61aabace33` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:43.910150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fd8ab-9ff3-40d2-9e19-1642729aadfe None admin] RBAC: Authorizing `identity:get_user(user_id=b0cca89fab4740169878ca61aabace33)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:43.911923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fd8ab-9ff3-40d2-9e19-1642729aadfe None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:43.913922 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 59/118] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:43 2022] GET /identity/v3/users/b0cca89fab4740169878ca61aabace33 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:43.915022 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fd8ab-9ff3-40d2-9e19-1642729aadfe None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:43.935933 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:43.973776 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0a435-9335-4443-84d9-09ac22ebc3e9 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:43.998446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:44.011478 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13094242-1005-46ad-97b8-412e6251e624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:44.012526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13094242-1005-46ad-97b8-412e6251e624 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:44.013068 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13094242-1005-46ad-97b8-412e6251e624 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:44.013554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13094242-1005-46ad-97b8-412e6251e624 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:44.025764 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13094242-1005-46ad-97b8-412e6251e624 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:44.028830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fd8ab-9ff3-40d2-9e19-1642729aadfe None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:44.031603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-13094242-1005-46ad-97b8-412e6251e624 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:13:44.031603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:44.031603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:44.031603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:44.031603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:44.031603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:44.031603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:44.031603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:44.031603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:44.031603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:44.031603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:44.031603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:44.031603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:44.031603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:44.031603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:44.031603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:44.032500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:44.032500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:44.032500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:44.032500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:44.032500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:44.032500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:44.032500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:44.032500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:44.032500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:44.032500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:44.032500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:44.032500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:44.032500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:44.032500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:44.032500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:44.032500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:44.033278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:44.033278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:44.033278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:44.033278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:44.033278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:44.033278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:44.033278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:44.033278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:44.033278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:44.033278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:44.033278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:13:44.033278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:13:44.033278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 60/119] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:13:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:44.034497 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09aae8-03e8-4625-95fc-57785154a18c None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:44.035111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09aae8-03e8-4625-95fc-57785154a18c None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:44.035504 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09aae8-03e8-4625-95fc-57785154a18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:44.053912 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-4f09aae8-03e8-4625-95fc-57785154a18c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:44.055147 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-4f09aae8-03e8-4625-95fc-57785154a18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:44.071842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63857c-5126-4a95-a33f-86aa83f87be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:44.073217 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63857c-5126-4a95-a33f-86aa83f87be0 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:44.073323 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63857c-5126-4a95-a33f-86aa83f87be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:44.073621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63857c-5126-4a95-a33f-86aa83f87be0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:44.074481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63857c-5126-4a95-a33f-86aa83f87be0 None admin] RBAC: Adding query filter params (role.id=9c013d602e9640b1883700191dcbd10a, scope.project.id=2269c4a7f991405497de468e3a9a745a, user.id=0bb1f8cff11d4177879e801bed0e8db4) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:44.074813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63857c-5126-4a95-a33f-86aa83f87be0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:44.076289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63857c-5126-4a95-a33f-86aa83f87be0 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:44.082212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 60/120] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:13:43 2022] GET /identity/v3/role_assignments?user.id=0bb1f8cff11d4177879e801bed0e8db4&scope.project.id=2269c4a7f991405497de468e3a9a745a&role.id=9c013d602e9640b1883700191dcbd10a => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:44.090997 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63857c-5126-4a95-a33f-86aa83f87be0 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:44.152571 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 61/121] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:44.153683 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09aae8-03e8-4625-95fc-57785154a18c None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:44.179798 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:44.201366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63857c-5126-4a95-a33f-86aa83f87be0 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:44.249543 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349cfb03-b2cd-4106-98ad-1a31a4eb8050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:44.250188 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cfb03-b2cd-4106-98ad-1a31a4eb8050 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:44.250472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cfb03-b2cd-4106-98ad-1a31a4eb8050 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:44.250750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cfb03-b2cd-4106-98ad-1a31a4eb8050 None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:44.251922 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349cfb03-b2cd-4106-98ad-1a31a4eb8050 None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:44.253452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349cfb03-b2cd-4106-98ad-1a31a4eb8050 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:44.255492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 61/122] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:44 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:44.256355 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349cfb03-b2cd-4106-98ad-1a31a4eb8050 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:44.275049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09aae8-03e8-4625-95fc-57785154a18c None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:44.315401 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33109bd-6c86-4822-8060-7fa8914dadd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:44.316381 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33109bd-6c86-4822-8060-7fa8914dadd0 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:44.316779 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33109bd-6c86-4822-8060-7fa8914dadd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:44.317158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33109bd-6c86-4822-8060-7fa8914dadd0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:44.329799 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33109bd-6c86-4822-8060-7fa8914dadd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:44.335233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-d33109bd-6c86-4822-8060-7fa8914dadd0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:13:44.335233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:44.335233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:44.335233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:44.335233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:44.335233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:44.335233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:44.335233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:44.335233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:44.335233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:44.335233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:44.335233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:44.335233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:44.335233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:44.335233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:44.335233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:44.336097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:44.336097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:44.336097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:44.336097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:44.336097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:44.336097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:44.336097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:44.336097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:44.336097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:44.336097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:44.336097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:44.336097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:44.336097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:44.336097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:44.336097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:44.336097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:44.337292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:44.337292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:44.337292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:44.337292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:44.337292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:44.337292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:44.337292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:44.337292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:44.337292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:44.337292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:44.337292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:13:44.337292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:13:44.337292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 62/123] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:44.338694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33109bd-6c86-4822-8060-7fa8914dadd0 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:44.341796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:44.388308 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349cfb03-b2cd-4106-98ad-1a31a4eb8050 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:44.457496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b7f1c-7357-4345-8933-bfd4aebc517a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:44.458054 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b7f1c-7357-4345-8933-bfd4aebc517a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:44.458343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b7f1c-7357-4345-8933-bfd4aebc517a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:44.458624 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b7f1c-7357-4345-8933-bfd4aebc517a None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:44.459335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b7f1c-7357-4345-8933-bfd4aebc517a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:44.459647 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b7f1c-7357-4345-8933-bfd4aebc517a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:44.461060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b7f1c-7357-4345-8933-bfd4aebc517a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:44.472244 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33109bd-6c86-4822-8060-7fa8914dadd0 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:44.478649 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 62/124] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:13:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:44.479545 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b7f1c-7357-4345-8933-bfd4aebc517a None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:44.500469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:44.515232 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b48b23-22b7-44eb-a8e1-db36a7a7d141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:44.515968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b48b23-22b7-44eb-a8e1-db36a7a7d141 None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:44.518601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b48b23-22b7-44eb-a8e1-db36a7a7d141 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:44.518601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b48b23-22b7-44eb-a8e1-db36a7a7d141 None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a/users/b0cca89fab4740169878ca61aabace33/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:44.518942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b48b23-22b7-44eb-a8e1-db36a7a7d141 None admin] RBAC: Authorizing `identity:create_grant(project_id=2269c4a7f991405497de468e3a9a745a, user_id=b0cca89fab4740169878ca61aabace33, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:44.520554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b48b23-22b7-44eb-a8e1-db36a7a7d141 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:44.605041 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 63/125] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:13:44 2022] PUT /identity/v3/projects/2269c4a7f991405497de468e3a9a745a/users/b0cca89fab4740169878ca61aabace33/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:13:44.605984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b48b23-22b7-44eb-a8e1-db36a7a7d141 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:44.651660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b7f1c-7357-4345-8933-bfd4aebc517a None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:44.662036 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:44.717229 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b581b8-f8d0-4b39-afb4-6ba13f3ee473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:44.717915 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b581b8-f8d0-4b39-afb4-6ba13f3ee473 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:44.718205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b581b8-f8d0-4b39-afb4-6ba13f3ee473 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:44.718487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b581b8-f8d0-4b39-afb4-6ba13f3ee473 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:44.719317 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b581b8-f8d0-4b39-afb4-6ba13f3ee473 None admin] RBAC: Adding query filter params (name=member) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:44.719639 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b581b8-f8d0-4b39-afb4-6ba13f3ee473 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:44.721094 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b581b8-f8d0-4b39-afb4-6ba13f3ee473 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:44.744177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 63/126] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:13:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:44.746927 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b581b8-f8d0-4b39-afb4-6ba13f3ee473 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:44.803670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b48b23-22b7-44eb-a8e1-db36a7a7d141 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:44.850714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237cac9-d9e0-40ca-b0e5-8781f7d73cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:44.851398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237cac9-d9e0-40ca-b0e5-8781f7d73cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:44.851689 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237cac9-d9e0-40ca-b0e5-8781f7d73cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:44.852226 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237cac9-d9e0-40ca-b0e5-8781f7d73cc9 None admin] PATH_INFO: `/v3/users/b0cca89fab4740169878ca61aabace33` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:44.854634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237cac9-d9e0-40ca-b0e5-8781f7d73cc9 None admin] RBAC: Authorizing `identity:get_user(user_id=b0cca89fab4740169878ca61aabace33)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:44.856516 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237cac9-d9e0-40ca-b0e5-8781f7d73cc9 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:44.859148 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:44.859279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 64/127] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:44 2022] GET /identity/v3/users/b0cca89fab4740169878ca61aabace33 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:44.866939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237cac9-d9e0-40ca-b0e5-8781f7d73cc9 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:44.907688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b581b8-f8d0-4b39-afb4-6ba13f3ee473 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:44.981437 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204ed44-e72e-468f-a4cd-b11fd4a9898b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:44.982002 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204ed44-e72e-468f-a4cd-b11fd4a9898b None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:44.982289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204ed44-e72e-468f-a4cd-b11fd4a9898b None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:44.982571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204ed44-e72e-468f-a4cd-b11fd4a9898b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:45.007697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204ed44-e72e-468f-a4cd-b11fd4a9898b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:45.013957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-e204ed44-e72e-468f-a4cd-b11fd4a9898b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:13:45.013957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:45.013957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:45.013957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:45.013957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:45.013957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:45.013957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:45.013957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:45.013957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:45.013957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:45.013957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:45.013957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:45.013957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:45.013957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:45.013957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:45.013957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:45.014838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:45.014838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:45.014838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:45.014838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:45.014838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:45.014838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:45.014838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:45.014838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:45.014838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:45.014838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:45.014838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:45.014838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:45.014838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:45.014838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:45.014838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:45.014838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:45.015640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:45.015640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:45.015640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:45.015640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:45.015640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:45.015640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:45.015640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:45.015640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:45.015640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:45.015640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:45.015640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:13:45.015640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:13:45.022812 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 64/128] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:13:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:45.023986 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204ed44-e72e-468f-a4cd-b11fd4a9898b None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:45.041816 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:45.046396 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237cac9-d9e0-40ca-b0e5-8781f7d73cc9 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:45.101306 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2c9f2-a8d3-4176-9d5a-bb837d2ff52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:45.102221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2c9f2-a8d3-4176-9d5a-bb837d2ff52b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:45.103511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2c9f2-a8d3-4176-9d5a-bb837d2ff52b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:45.106014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2c9f2-a8d3-4176-9d5a-bb837d2ff52b None admin] PATH_INFO: `/v3/users/b0cca89fab4740169878ca61aabace33` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:45.108092 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f2c9f2-a8d3-4176-9d5a-bb837d2ff52b None admin] RBAC: Authorizing `identity:get_user(user_id=b0cca89fab4740169878ca61aabace33)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:45.109829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f2c9f2-a8d3-4176-9d5a-bb837d2ff52b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:45.113040 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 65/129] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:44 2022] GET /identity/v3/users/b0cca89fab4740169878ca61aabace33 => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:45.116490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2c9f2-a8d3-4176-9d5a-bb837d2ff52b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:45.223148 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204ed44-e72e-468f-a4cd-b11fd4a9898b None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:45.263873 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:45.273157 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b2e58-3790-4ee4-945b-9132736b7579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:45.274107 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b2e58-3790-4ee4-945b-9132736b7579 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:45.275877 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b2e58-3790-4ee4-945b-9132736b7579 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:45.277964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b2e58-3790-4ee4-945b-9132736b7579 None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:45.279559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b2e58-3790-4ee4-945b-9132736b7579 None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:45.283242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b2e58-3790-4ee4-945b-9132736b7579 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:45.288970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 65/130] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:45 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:45.292513 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b2e58-3790-4ee4-945b-9132736b7579 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:45.319925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2c9f2-a8d3-4176-9d5a-bb837d2ff52b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:45.368804 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2778f4-e3a6-47bf-9339-c18c5ecfe6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:45.371529 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2778f4-e3a6-47bf-9339-c18c5ecfe6bf None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:45.371995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2778f4-e3a6-47bf-9339-c18c5ecfe6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:45.372456 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2778f4-e3a6-47bf-9339-c18c5ecfe6bf None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:45.373304 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2778f4-e3a6-47bf-9339-c18c5ecfe6bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:45.374830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2778f4-e3a6-47bf-9339-c18c5ecfe6bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:45.377122 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2778f4-e3a6-47bf-9339-c18c5ecfe6bf None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:45.391386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 66/131] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:45.393222 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2778f4-e3a6-47bf-9339-c18c5ecfe6bf None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:45.428775 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:45.508113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b2e58-3790-4ee4-945b-9132736b7579 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:45.529849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2778f4-e3a6-47bf-9339-c18c5ecfe6bf None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:45.554681 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16228170-8d4d-4661-b471-5fba53f9de86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:45.556056 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16228170-8d4d-4661-b471-5fba53f9de86 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:45.556786 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16228170-8d4d-4661-b471-5fba53f9de86 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:45.557298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16228170-8d4d-4661-b471-5fba53f9de86 None admin] PATH_INFO: `/v3/projects/4a0c087d2d50474e856da494b5d6563b` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:45.559317 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16228170-8d4d-4661-b471-5fba53f9de86 None admin] RBAC: Authorizing `identity:get_project(project_id=4a0c087d2d50474e856da494b5d6563b)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:45.561244 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16228170-8d4d-4661-b471-5fba53f9de86 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:45.563818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 66/132] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:45 2022] GET /identity/v3/projects/4a0c087d2d50474e856da494b5d6563b => generated 312 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:45.565025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16228170-8d4d-4661-b471-5fba53f9de86 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:45.570274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2321358-36e2-460c-b91f-f19e779716f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:45.570600 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2321358-36e2-460c-b91f-f19e779716f1 None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:45.571004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2321358-36e2-460c-b91f-f19e779716f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:45.571408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2321358-36e2-460c-b91f-f19e779716f1 None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a/users/b0cca89fab4740169878ca61aabace33/roles/1182ffc3e9f9477db2003500b7bfd402` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:45.574387 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2321358-36e2-460c-b91f-f19e779716f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=2269c4a7f991405497de468e3a9a745a, user_id=b0cca89fab4740169878ca61aabace33, role_id=1182ffc3e9f9477db2003500b7bfd402)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:45.576065 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2321358-36e2-460c-b91f-f19e779716f1 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:45.639127 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:45.672840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16228170-8d4d-4661-b471-5fba53f9de86 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:45.706758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b38d7e-3929-4c99-b89f-7c57b860dc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:45.707496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b38d7e-3929-4c99-b89f-7c57b860dc67 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:45.707787 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b38d7e-3929-4c99-b89f-7c57b860dc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:45.708071 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b38d7e-3929-4c99-b89f-7c57b860dc67 None admin] PATH_INFO: `/v3/users/40eedd635c074248bf2c6c67ebf16bb0` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:45.714144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b38d7e-3929-4c99-b89f-7c57b860dc67 None admin] RBAC: Authorizing `identity:get_user(user_id=40eedd635c074248bf2c6c67ebf16bb0)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:45.714144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b38d7e-3929-4c99-b89f-7c57b860dc67 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:45.714144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 67/134] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:45 2022] GET /identity/v3/users/40eedd635c074248bf2c6c67ebf16bb0 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:45.718487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d99fdd-ff9e-4b21-91c9-65b69eadd9a4 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:45.718487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d99fdd-ff9e-4b21-91c9-65b69eadd9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:45.718487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d99fdd-ff9e-4b21-91c9-65b69eadd9a4 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:45.718696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 67/134] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:13:45 2022] PUT /identity/v3/projects/2269c4a7f991405497de468e3a9a745a/users/b0cca89fab4740169878ca61aabace33/roles/1182ffc3e9f9477db2003500b7bfd402 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:13:45.718696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2321358-36e2-460c-b91f-f19e779716f1 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:45.719198 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 68/135] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:45.723621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187e72e-b5d6-4c9f-86ab-319c61c4314f None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:45.723917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187e72e-b5d6-4c9f-86ab-319c61c4314f None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:45.724212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187e72e-b5d6-4c9f-86ab-319c61c4314f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:45.758766 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-b187e72e-b5d6-4c9f-86ab-319c61c4314f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:45.761652 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b187e72e-b5d6-4c9f-86ab-319c61c4314f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:45.914013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 69/136] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:45.914830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187e72e-b5d6-4c9f-86ab-319c61c4314f None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:45.930040 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:45.949608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2321358-36e2-460c-b91f-f19e779716f1 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:46.004358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:46.036519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fcbd2-dd55-4c19-96e4-29533a8a6195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:46.037186 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fcbd2-dd55-4c19-96e4-29533a8a6195 None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:46.037471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fcbd2-dd55-4c19-96e4-29533a8a6195 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:46.037747 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fcbd2-dd55-4c19-96e4-29533a8a6195 None admin] PATH_INFO: `/v3/projects/4a0c087d2d50474e856da494b5d6563b/users/b0cca89fab4740169878ca61aabace33/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:46.039728 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fcbd2-dd55-4c19-96e4-29533a8a6195 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0c087d2d50474e856da494b5d6563b, user_id=b0cca89fab4740169878ca61aabace33, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:46.041786 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fcbd2-dd55-4c19-96e4-29533a8a6195 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:46.046192 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187e72e-b5d6-4c9f-86ab-319c61c4314f None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:46.100369 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a12ce-d182-4887-9829-948d5553c8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:46.101144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a12ce-d182-4887-9829-948d5553c8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:46.101573 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a12ce-d182-4887-9829-948d5553c8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:46.101921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a12ce-d182-4887-9829-948d5553c8b7 None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:46.103103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a12ce-d182-4887-9829-948d5553c8b7 None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:46.105400 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a12ce-d182-4887-9829-948d5553c8b7 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:46.107318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 70/137] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:45 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:46.112977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a12ce-d182-4887-9829-948d5553c8b7 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:46.126020 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 68/138] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:13:45 2022] PUT /identity/v3/projects/4a0c087d2d50474e856da494b5d6563b/users/b0cca89fab4740169878ca61aabace33/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:13:46.165684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776dbc2-5c29-485b-8dd4-6a114e7f8992 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:46.165882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776dbc2-5c29-485b-8dd4-6a114e7f8992 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:46.166727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776dbc2-5c29-485b-8dd4-6a114e7f8992 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:46.168117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 69/139] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:13:46.178510 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c7b28-8a25-4db7-b31f-3da557c0bf55 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:46.178815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c7b28-8a25-4db7-b31f-3da557c0bf55 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:46.179166 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c7b28-8a25-4db7-b31f-3da557c0bf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:46.212989 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-f42c7b28-8a25-4db7-b31f-3da557c0bf55 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:46.214364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f42c7b28-8a25-4db7-b31f-3da557c0bf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:46.286824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a12ce-d182-4887-9829-948d5553c8b7 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:46.338353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b64d9-0221-4689-b908-091df65106ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:46.341774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b64d9-0221-4689-b908-091df65106ef None admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:46.342014 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b64d9-0221-4689-b908-091df65106ef None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:46.342292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b64d9-0221-4689-b908-091df65106ef None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a/users/40eedd635c074248bf2c6c67ebf16bb0/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:46.348881 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b64d9-0221-4689-b908-091df65106ef None admin] RBAC: Authorizing `identity:create_grant(project_id=2269c4a7f991405497de468e3a9a745a, user_id=40eedd635c074248bf2c6c67ebf16bb0, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:46.352361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b64d9-0221-4689-b908-091df65106ef None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:46.360711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 70/140] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:46.361732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c7b28-8a25-4db7-b31f-3da557c0bf55 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:46.365781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:46.447939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 71/141] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:13:46 2022] PUT /identity/v3/projects/2269c4a7f991405497de468e3a9a745a/users/40eedd635c074248bf2c6c67ebf16bb0/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:13:46.453574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:46.477051 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c7b28-8a25-4db7-b31f-3da557c0bf55 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:46.515960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30a9f7-c15e-4e12-a95d-554545aa48d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:46.516620 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30a9f7-c15e-4e12-a95d-554545aa48d9 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:46.516908 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30a9f7-c15e-4e12-a95d-554545aa48d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:46.517188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30a9f7-c15e-4e12-a95d-554545aa48d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:46.518404 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30a9f7-c15e-4e12-a95d-554545aa48d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:46.519979 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30a9f7-c15e-4e12-a95d-554545aa48d9 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:46.521673 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 71/142] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:13:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:46.527995 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b64d9-0221-4689-b908-091df65106ef None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:46.696268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b64d9-0221-4689-b908-091df65106ef None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:46.706930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30a9f7-c15e-4e12-a95d-554545aa48d9 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:46.764698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f79898-da28-4184-b8fa-394765565392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:46.765698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f79898-da28-4184-b8fa-394765565392 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:46.766194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f79898-da28-4184-b8fa-394765565392 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:46.766667 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f79898-da28-4184-b8fa-394765565392 None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:46.767549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f79898-da28-4184-b8fa-394765565392 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:46.769223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f79898-da28-4184-b8fa-394765565392 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:46.844613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30a9f7-c15e-4e12-a95d-554545aa48d9 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:46.883594 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5d0be-1d18-453f-b317-3fd23788a105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:46.891471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 72/143] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:13:46 2022] POST /identity/v3/users => generated 283 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:13:46.891997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5d0be-1d18-453f-b317-3fd23788a105 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:46.892494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5d0be-1d18-453f-b317-3fd23788a105 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:46.893025 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5d0be-1d18-453f-b317-3fd23788a105 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:46.910087 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5d0be-1d18-453f-b317-3fd23788a105 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:46.915599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-13b5d0be-1d18-453f-b317-3fd23788a105 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:13:46.915599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:46.915599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:46.915599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:46.915599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:46.915599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:46.915599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:46.915599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:46.915599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:46.915599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:46.915599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:46.915599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:46.915599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:46.915599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:46.915599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:46.915599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:46.916583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:46.916583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:46.916583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:46.916583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:46.916583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:46.916583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:46.916583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:46.916583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:46.916583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:46.916583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:46.916583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:46.916583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:46.916583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:46.916583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:46.916583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:46.916583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:46.917361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:46.917361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:46.917361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:46.917361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:46.917361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:46.917361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:46.917361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:46.917361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:46.917361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:46.917361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:46.917361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:13:46.917361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:13:46.920078 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 72/144] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:13:46.926129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f79898-da28-4184-b8fa-394765565392 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:47.003088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:47.069575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f79898-da28-4184-b8fa-394765565392 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:47.123705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14ab0f-c9e9-4d00-8c6e-cff979f13694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:47.124385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14ab0f-c9e9-4d00-8c6e-cff979f13694 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:47.124680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14ab0f-c9e9-4d00-8c6e-cff979f13694 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:47.124957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14ab0f-c9e9-4d00-8c6e-cff979f13694 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:47.125662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14ab0f-c9e9-4d00-8c6e-cff979f13694 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:47.125968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14ab0f-c9e9-4d00-8c6e-cff979f13694 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:47.127526 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14ab0f-c9e9-4d00-8c6e-cff979f13694 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:47.139449 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 73/145] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:13:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:47.145235 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5d0be-1d18-453f-b317-3fd23788a105 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:47.163261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:47.276502 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5d0be-1d18-453f-b317-3fd23788a105 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:47.323324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f9e4a-d976-4bbf-8a37-7d0d7ab73e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:47.324018 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9e4a-d976-4bbf-8a37-7d0d7ab73e9e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:47.324308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9e4a-d976-4bbf-8a37-7d0d7ab73e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:47.324591 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9e4a-d976-4bbf-8a37-7d0d7ab73e9e None admin] PATH_INFO: `/v3/users/0bb1f8cff11d4177879e801bed0e8db4` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:47.325849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f9e4a-d976-4bbf-8a37-7d0d7ab73e9e None admin] RBAC: Authorizing `identity:get_user(user_id=0bb1f8cff11d4177879e801bed0e8db4)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:47.327353 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f9e4a-d976-4bbf-8a37-7d0d7ab73e9e None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:47.328948 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 73/146] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:47 2022] GET /identity/v3/users/0bb1f8cff11d4177879e801bed0e8db4 => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:47.335114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14ab0f-c9e9-4d00-8c6e-cff979f13694 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:47.393627 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9c8c7-cdf9-4841-8cdb-a6a93bc7b657 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:47.393916 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9c8c7-cdf9-4841-8cdb-a6a93bc7b657 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:47.394205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9c8c7-cdf9-4841-8cdb-a6a93bc7b657 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:47.395829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 74/147] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:47.401474 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1af804-5d30-448e-9e27-393c6ea6895e None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:47.401770 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1af804-5d30-448e-9e27-393c6ea6895e None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:47.402053 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1af804-5d30-448e-9e27-393c6ea6895e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:47.420975 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-af1af804-5d30-448e-9e27-393c6ea6895e None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:47.422191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-af1af804-5d30-448e-9e27-393c6ea6895e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:47.523187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14ab0f-c9e9-4d00-8c6e-cff979f13694 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:47.549252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:47.562936 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 75/148] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:13:47.590484 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c421a91-5747-4bba-9c39-3c9cd9f20bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:47.591517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c421a91-5747-4bba-9c39-3c9cd9f20bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:47.591805 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c421a91-5747-4bba-9c39-3c9cd9f20bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:47.592087 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c421a91-5747-4bba-9c39-3c9cd9f20bd0 None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:47.593259 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c421a91-5747-4bba-9c39-3c9cd9f20bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:47.594728 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c421a91-5747-4bba-9c39-3c9cd9f20bd0 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:47.596702 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 74/149] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:47 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:47.602434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1af804-5d30-448e-9e27-393c6ea6895e None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:47.616463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:47.746099 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1af804-5d30-448e-9e27-393c6ea6895e None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:47.788869 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c3194-1301-4392-b250-ef87806f62a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:47.789543 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c3194-1301-4392-b250-ef87806f62a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:47.789825 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c3194-1301-4392-b250-ef87806f62a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:47.790104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c3194-1301-4392-b250-ef87806f62a9 None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a/users/0bb1f8cff11d4177879e801bed0e8db4/roles/9c013d602e9640b1883700191dcbd10a` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:47.795533 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c3194-1301-4392-b250-ef87806f62a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2269c4a7f991405497de468e3a9a745a, user_id=0bb1f8cff11d4177879e801bed0e8db4, role_id=9c013d602e9640b1883700191dcbd10a)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:47.797002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c3194-1301-4392-b250-ef87806f62a9 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:47.874984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 76/150] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:13:47 2022] PUT /identity/v3/projects/2269c4a7f991405497de468e3a9a745a/users/0bb1f8cff11d4177879e801bed0e8db4/roles/9c013d602e9640b1883700191dcbd10a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:13:48.185327 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c421a91-5747-4bba-9c39-3c9cd9f20bd0 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:48.307013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c421a91-5747-4bba-9c39-3c9cd9f20bd0 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:48.348804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb337ea8-08df-4742-b340-532a281648d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:48.349454 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb337ea8-08df-4742-b340-532a281648d1 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:48.349741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb337ea8-08df-4742-b340-532a281648d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:48.350017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb337ea8-08df-4742-b340-532a281648d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:48.365304 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb337ea8-08df-4742-b340-532a281648d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:48.370825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-fb337ea8-08df-4742-b340-532a281648d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:13:48.370825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:48.370825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:48.370825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:48.370825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:48.370825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:48.370825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:48.370825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:48.370825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:48.370825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:48.370825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:48.370825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:48.370825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:48.370825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:48.370825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:48.370825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:48.371671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:48.371671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:48.371671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:48.371671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:48.371671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:48.371671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:48.371671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:48.371671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:48.371671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:48.371671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:48.371671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:48.371671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:48.371671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:48.371671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:48.371671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:48.371671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:48.372832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:48.372832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:48.372832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:48.372832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:48.372832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:48.372832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:48.372832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:48.372832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:48.372832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:48.372832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:48.372832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:13:48.372832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:13:48.372832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 75/151] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:48.377793 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c3194-1301-4392-b250-ef87806f62a9 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:48.482705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:48.529838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c3194-1301-4392-b250-ef87806f62a9 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:48.575254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10226f-ba57-4678-a03b-b9740b22d9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:48.575884 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10226f-ba57-4678-a03b-b9740b22d9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:48.576168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10226f-ba57-4678-a03b-b9740b22d9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:48.576442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10226f-ba57-4678-a03b-b9740b22d9c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:48.577188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10226f-ba57-4678-a03b-b9740b22d9c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:48.577491 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10226f-ba57-4678-a03b-b9740b22d9c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:48.578970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10226f-ba57-4678-a03b-b9740b22d9c5 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:48.592091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 77/152] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:13:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:48.599040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb337ea8-08df-4742-b340-532a281648d1 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:48.680971 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:48.738898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3555c1-18a3-41e8-95a2-e1b526eb3d4a None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:48.739057 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3555c1-18a3-41e8-95a2-e1b526eb3d4a None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:48.739349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3555c1-18a3-41e8-95a2-e1b526eb3d4a None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:48.740692 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 78/153] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:13:48.748034 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb337ea8-08df-4742-b340-532a281648d1 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:48.748612 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca0429-f23a-4191-9682-6652bc0f5238 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:48.748907 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca0429-f23a-4191-9682-6652bc0f5238 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:48.749188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca0429-f23a-4191-9682-6652bc0f5238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:48.790832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-e7ca0429-f23a-4191-9682-6652bc0f5238 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:48.792150 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e7ca0429-f23a-4191-9682-6652bc0f5238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:48.833388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19ba34-b551-46e3-91d3-79e13d171a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:48.834043 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19ba34-b551-46e3-91d3-79e13d171a8e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:48.834328 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19ba34-b551-46e3-91d3-79e13d171a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:48.834605 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19ba34-b551-46e3-91d3-79e13d171a8e None admin] PATH_INFO: `/v3/users/b0cca89fab4740169878ca61aabace33` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:48.839499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19ba34-b551-46e3-91d3-79e13d171a8e None admin] RBAC: Authorizing `identity:get_user(user_id=b0cca89fab4740169878ca61aabace33)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:48.840978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19ba34-b551-46e3-91d3-79e13d171a8e None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:48.842967 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 76/154] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:48 2022] GET /identity/v3/users/b0cca89fab4740169878ca61aabace33 => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:48.848139 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19ba34-b551-46e3-91d3-79e13d171a8e None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:48.923898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 79/155] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 05 14:13:48.987691 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19ba34-b551-46e3-91d3-79e13d171a8e None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:49.003221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:49.031313 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8342c-d9d4-4788-a98f-1217fc3d0501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:49.032002 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8342c-d9d4-4788-a98f-1217fc3d0501 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:49.032369 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8342c-d9d4-4788-a98f-1217fc3d0501 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:49.032705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8342c-d9d4-4788-a98f-1217fc3d0501 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:49.130291 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8342c-d9d4-4788-a98f-1217fc3d0501 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:49.131828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8342c-d9d4-4788-a98f-1217fc3d0501 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:49.259877 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 77/156] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:13:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 14:13:49.265464 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca0429-f23a-4191-9682-6652bc0f5238 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:49.336695 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:49.383986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca0429-f23a-4191-9682-6652bc0f5238 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:49.420885 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b08ee7-a8fb-4efa-9a23-79cd53a3eaec None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:49.421198 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b08ee7-a8fb-4efa-9a23-79cd53a3eaec None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:49.421440 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b08ee7-a8fb-4efa-9a23-79cd53a3eaec None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:49.422820 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 78/157] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:13:49.427929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d65cb-ccf3-407f-9717-81d052b3f69a None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:49.428218 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d65cb-ccf3-407f-9717-81d052b3f69a None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:49.428496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d65cb-ccf3-407f-9717-81d052b3f69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:49.442484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e61bdc-b9f3-46e8-86a8-f3f374536b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:49.443220 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e61bdc-b9f3-46e8-86a8-f3f374536b9c None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:49.443511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e61bdc-b9f3-46e8-86a8-f3f374536b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:49.443787 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e61bdc-b9f3-46e8-86a8-f3f374536b9c None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:49.445968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e61bdc-b9f3-46e8-86a8-f3f374536b9c None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:49.447500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e61bdc-b9f3-46e8-86a8-f3f374536b9c None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:49.449158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 80/158] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:49 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:49.453471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-ca4d65cb-ccf3-407f-9717-81d052b3f69a None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:49.455395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e61bdc-b9f3-46e8-86a8-f3f374536b9c None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:49.456123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ca4d65cb-ccf3-407f-9717-81d052b3f69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:49.458399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:49.559820 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 79/159] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:13:49.560606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d65cb-ccf3-407f-9717-81d052b3f69a None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:49.625618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e61bdc-b9f3-46e8-86a8-f3f374536b9c None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:49.655967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:49.661605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbd0a7-3e30-45e9-ab21-0049abb6d322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:49.662230 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbd0a7-3e30-45e9-ab21-0049abb6d322 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:49.662515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbd0a7-3e30-45e9-ab21-0049abb6d322 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:49.662818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbd0a7-3e30-45e9-ab21-0049abb6d322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:49.663548 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbd0a7-3e30-45e9-ab21-0049abb6d322 None admin] RBAC: Adding query filter params (role.id=5812e6538f3f4bdfbfbe11cd25c02957, scope.project.id=2269c4a7f991405497de468e3a9a745a, user.id=b0cca89fab4740169878ca61aabace33) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:49.663862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbd0a7-3e30-45e9-ab21-0049abb6d322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:49.665306 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbd0a7-3e30-45e9-ab21-0049abb6d322 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:49.672353 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 81/160] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:13:49 2022] GET /identity/v3/role_assignments?user.id=b0cca89fab4740169878ca61aabace33&scope.project.id=2269c4a7f991405497de468e3a9a745a&role.id=5812e6538f3f4bdfbfbe11cd25c02957 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:13:49.673563 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd62f0-5263-40a4-aa01-234f60178128 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:49.673856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd62f0-5263-40a4-aa01-234f60178128 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:49.674134 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd62f0-5263-40a4-aa01-234f60178128 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:49.678092 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 82/161] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:49.683608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95425a8c-6e32-4020-b821-5c159e25c076 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:49.683817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95425a8c-6e32-4020-b821-5c159e25c076 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:49.684095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95425a8c-6e32-4020-b821-5c159e25c076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:49.703818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-95425a8c-6e32-4020-b821-5c159e25c076 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:49.705103 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-95425a8c-6e32-4020-b821-5c159e25c076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:49.740351 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d65cb-ccf3-407f-9717-81d052b3f69a None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:49.781202 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 83/162] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:13:49.821216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35496e-9bd3-49ab-a2ed-1ce127c22d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:49.821793 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35496e-9bd3-49ab-a2ed-1ce127c22d91 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:49.822076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35496e-9bd3-49ab-a2ed-1ce127c22d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:49.822900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35496e-9bd3-49ab-a2ed-1ce127c22d91 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:49.837577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:49.847108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35496e-9bd3-49ab-a2ed-1ce127c22d91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:49.852005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-7c35496e-9bd3-49ab-a2ed-1ce127c22d91 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:13:49.852005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:49.852005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:49.852005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:49.852005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:49.852005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:49.852005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:49.852005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:49.852005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:49.852005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:49.852005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:49.852005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:49.852005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:49.852005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:49.852005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:49.852005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:49.852979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:49.852979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:49.852979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:49.852979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:49.852979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:49.852979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:49.852979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:49.852979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:49.852979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:49.852979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:49.852979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:49.852979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:49.852979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:49.852979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:49.852979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:49.852979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:49.853807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:49.853807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:49.853807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:49.853807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:49.853807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:49.853807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:49.853807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:49.853807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:49.853807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:49.853807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:49.853807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:13:49.853807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:13:49.853807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 80/163] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:13:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:49.861152 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95425a8c-6e32-4020-b821-5c159e25c076 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:50.000503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95425a8c-6e32-4020-b821-5c159e25c076 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:50.022455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:50.038083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71838c-5187-4031-96af-6afcfecbfc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:50.038709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71838c-5187-4031-96af-6afcfecbfc89 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:50.039016 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71838c-5187-4031-96af-6afcfecbfc89 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:50.039293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71838c-5187-4031-96af-6afcfecbfc89 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:50.039976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e71838c-5187-4031-96af-6afcfecbfc89 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:50.040281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e71838c-5187-4031-96af-6afcfecbfc89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:50.042201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e71838c-5187-4031-96af-6afcfecbfc89 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:50.051588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 84/164] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:13:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:50.059867 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35496e-9bd3-49ab-a2ed-1ce127c22d91 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:50.214815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35496e-9bd3-49ab-a2ed-1ce127c22d91 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:50.254449 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1912a3-5f2e-4cb9-860d-f9df8cd7946d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:50.255108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1912a3-5f2e-4cb9-860d-f9df8cd7946d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:50.255386 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1912a3-5f2e-4cb9-860d-f9df8cd7946d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:50.255659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1912a3-5f2e-4cb9-860d-f9df8cd7946d None admin] PATH_INFO: `/v3/users/b0cca89fab4740169878ca61aabace33` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:50.256805 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1912a3-5f2e-4cb9-860d-f9df8cd7946d None admin] RBAC: Authorizing `identity:get_user(user_id=b0cca89fab4740169878ca61aabace33)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:50.258248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1912a3-5f2e-4cb9-860d-f9df8cd7946d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:50.260085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 81/165] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:50 2022] GET /identity/v3/users/b0cca89fab4740169878ca61aabace33 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:50.266570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71838c-5187-4031-96af-6afcfecbfc89 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:50.299383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1912a3-5f2e-4cb9-860d-f9df8cd7946d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:50.420681 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:50.458256 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71838c-5187-4031-96af-6afcfecbfc89 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:50.484672 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1912a3-5f2e-4cb9-860d-f9df8cd7946d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:50.516166 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3e958-3f02-48c5-a555-c0194ca2fb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:50.516781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3e958-3f02-48c5-a555-c0194ca2fb85 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:50.517058 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3e958-3f02-48c5-a555-c0194ca2fb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:50.517327 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3e958-3f02-48c5-a555-c0194ca2fb85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:50.548887 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9bca0c-c3fa-4c3c-926f-fe76a92d7ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:50.549518 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bca0c-c3fa-4c3c-926f-fe76a92d7ade None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:50.549794 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bca0c-c3fa-4c3c-926f-fe76a92d7ade None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:50.550061 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bca0c-c3fa-4c3c-926f-fe76a92d7ade None admin] PATH_INFO: `/v3/roles/member` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:50.567145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9bca0c-c3fa-4c3c-926f-fe76a92d7ade None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:50.574171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-6d9bca0c-c3fa-4c3c-926f-fe76a92d7ade None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:13:50.574171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:50.574171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:50.574171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:50.574171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:50.574171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:50.574171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:50.574171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:50.574171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:50.574171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:50.574171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:50.574171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:50.574171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:50.574171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:50.574171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:50.574171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:50.575205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:50.575205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:50.575205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:50.575205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:50.575205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:50.575205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:50.575205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:50.575205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:50.575205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:50.575205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:50.575205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:50.575205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:50.575205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:50.575205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:50.575205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:50.575205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:50.576168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:50.576168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:50.576168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:50.576168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:50.576168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:50.576168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:50.576168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:50.576168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:50.576168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:50.576168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:50.576168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:13:50.576168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:13:50.578500 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 82/166] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:13:50.580170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9bca0c-c3fa-4c3c-926f-fe76a92d7ade None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:50.606114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:50.632157 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3e958-3f02-48c5-a555-c0194ca2fb85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:50.633612 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3e958-3f02-48c5-a555-c0194ca2fb85 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:50.771442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 85/167] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:13:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:13:50.772547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3e958-3f02-48c5-a555-c0194ca2fb85 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:50.774065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9bca0c-c3fa-4c3c-926f-fe76a92d7ade None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:50.828446 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:50.848631 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6810716-8ec9-4281-b5f4-0b5f9bf8dda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:50.850144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6810716-8ec9-4281-b5f4-0b5f9bf8dda5 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:50.850451 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6810716-8ec9-4281-b5f4-0b5f9bf8dda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:50.850728 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6810716-8ec9-4281-b5f4-0b5f9bf8dda5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:50.874280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6810716-8ec9-4281-b5f4-0b5f9bf8dda5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:50.879970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-d6810716-8ec9-4281-b5f4-0b5f9bf8dda5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:13:50.879970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:50.879970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:50.879970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:50.879970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:50.879970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:50.879970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:50.879970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:50.879970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:50.879970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:50.879970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:50.879970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:50.879970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:50.879970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:50.879970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:50.879970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:50.880898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:50.880898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:50.880898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:50.880898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:50.880898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:50.880898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:50.880898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:50.880898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:50.880898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:50.880898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:50.880898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:50.880898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:50.880898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:50.880898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:50.880898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:50.880898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:50.881791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:50.881791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:50.881791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:50.881791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:50.881791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:50.881791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:50.881791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:50.881791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:50.881791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:50.881791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:50.881791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:13:50.881791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:13:50.881791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 83/168] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:13:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:50.882455 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6810716-8ec9-4281-b5f4-0b5f9bf8dda5 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:50.954518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:50.984086 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3e958-3f02-48c5-a555-c0194ca2fb85 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:51.020628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcca8b5-a483-48c2-9795-7f2477810e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:51.022108 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcca8b5-a483-48c2-9795-7f2477810e6e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:51.022827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcca8b5-a483-48c2-9795-7f2477810e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:51.023319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcca8b5-a483-48c2-9795-7f2477810e6e None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:51.024188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcca8b5-a483-48c2-9795-7f2477810e6e None admin] RBAC: Adding query filter params (name=member) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:51.024702 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcca8b5-a483-48c2-9795-7f2477810e6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:51.026352 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcca8b5-a483-48c2-9795-7f2477810e6e None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:51.035931 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 86/169] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:13:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:51.037073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcca8b5-a483-48c2-9795-7f2477810e6e None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:51.086589 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6810716-8ec9-4281-b5f4-0b5f9bf8dda5 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:51.141981 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcca8b5-a483-48c2-9795-7f2477810e6e None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:51.164460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0272d-b20b-4940-9c06-15b75821d2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:51.165113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0272d-b20b-4940-9c06-15b75821d2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:51.165397 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0272d-b20b-4940-9c06-15b75821d2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:51.165672 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0272d-b20b-4940-9c06-15b75821d2b0 None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:51.166999 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0272d-b20b-4940-9c06-15b75821d2b0 None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:51.168846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0272d-b20b-4940-9c06-15b75821d2b0 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:51.170665 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 84/170] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:50 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:51.171678 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0272d-b20b-4940-9c06-15b75821d2b0 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:51.185213 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3f571-be6d-442c-899a-4369c0c6a857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:51.186187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3f571-be6d-442c-899a-4369c0c6a857 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:51.186744 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3f571-be6d-442c-899a-4369c0c6a857 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:51.187261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3f571-be6d-442c-899a-4369c0c6a857 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:51.188159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3f571-be6d-442c-899a-4369c0c6a857 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:51.188672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3f571-be6d-442c-899a-4369c0c6a857 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:51.190537 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3f571-be6d-442c-899a-4369c0c6a857 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:51.200502 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 87/171] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:51.201605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3f571-be6d-442c-899a-4369c0c6a857 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:51.215782 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:51.328646 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:51.339309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3f571-be6d-442c-899a-4369c0c6a857 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:51.401355 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671e9bc-6afc-449b-9b6d-31ff64db2c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:51.402368 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671e9bc-6afc-449b-9b6d-31ff64db2c0a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:51.402896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671e9bc-6afc-449b-9b6d-31ff64db2c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:51.403383 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671e9bc-6afc-449b-9b6d-31ff64db2c0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:51.404318 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671e9bc-6afc-449b-9b6d-31ff64db2c0a None admin] RBAC: Adding query filter params (role.id=1182ffc3e9f9477db2003500b7bfd402, scope.project.id=2269c4a7f991405497de468e3a9a745a, user.id=b0cca89fab4740169878ca61aabace33) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:51.404845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671e9bc-6afc-449b-9b6d-31ff64db2c0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:51.406494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671e9bc-6afc-449b-9b6d-31ff64db2c0a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:51.421152 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 88/172] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:13:51 2022] GET /identity/v3/role_assignments?user.id=b0cca89fab4740169878ca61aabace33&scope.project.id=2269c4a7f991405497de468e3a9a745a&role.id=1182ffc3e9f9477db2003500b7bfd402 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:51.422344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671e9bc-6afc-449b-9b6d-31ff64db2c0a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:51.428941 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0272d-b20b-4940-9c06-15b75821d2b0 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:51.519620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8996b50-060a-4523-8b03-a73d89dac82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:51.520289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8996b50-060a-4523-8b03-a73d89dac82c None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:51.520570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8996b50-060a-4523-8b03-a73d89dac82c None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:51.520846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8996b50-060a-4523-8b03-a73d89dac82c None admin] PATH_INFO: `/v3/users/b0cca89fab4740169878ca61aabace33` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:51.523227 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8996b50-060a-4523-8b03-a73d89dac82c None admin] RBAC: Authorizing `identity:get_user(user_id=b0cca89fab4740169878ca61aabace33)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:51.524733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8996b50-060a-4523-8b03-a73d89dac82c None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:51.530815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 85/173] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:51 2022] GET /identity/v3/users/b0cca89fab4740169878ca61aabace33 => generated 275 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:51.535655 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8996b50-060a-4523-8b03-a73d89dac82c None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:51.540785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:51.612696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671e9bc-6afc-449b-9b6d-31ff64db2c0a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:51.675417 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8996b50-060a-4523-8b03-a73d89dac82c None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:51.684957 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af39382-78c9-4b54-aed3-00fdde9f8cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:51.685622 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af39382-78c9-4b54-aed3-00fdde9f8cda None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:51.685908 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af39382-78c9-4b54-aed3-00fdde9f8cda None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:51.686178 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af39382-78c9-4b54-aed3-00fdde9f8cda None admin] PATH_INFO: `/v3/users/40eedd635c074248bf2c6c67ebf16bb0` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:51.688305 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af39382-78c9-4b54-aed3-00fdde9f8cda None admin] RBAC: Authorizing `identity:get_user(user_id=40eedd635c074248bf2c6c67ebf16bb0)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:51.690038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af39382-78c9-4b54-aed3-00fdde9f8cda None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:51.692189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 89/174] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:51 2022] GET /identity/v3/users/40eedd635c074248bf2c6c67ebf16bb0 => generated 247 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:51.693339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870ec58-cad2-4857-b538-d51b8c824141 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:51.693620 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870ec58-cad2-4857-b538-d51b8c824141 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:51.693894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870ec58-cad2-4857-b538-d51b8c824141 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:51.699397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 90/175] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:51.700250 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870ec58-cad2-4857-b538-d51b8c824141 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:51.702040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:51.715378 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f59075-29fe-4144-8a84-34e435fda24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:51.716115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f59075-29fe-4144-8a84-34e435fda24d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:51.716450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f59075-29fe-4144-8a84-34e435fda24d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:51.716733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f59075-29fe-4144-8a84-34e435fda24d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:51.790205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f59075-29fe-4144-8a84-34e435fda24d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:51.792425 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f59075-29fe-4144-8a84-34e435fda24d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:51.907867 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870ec58-cad2-4857-b538-d51b8c824141 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:51.914575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 86/176] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:13:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:13:51.915726 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2e40d-4edb-43a2-9caf-6a5b0b699ea5 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:51.916141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2e40d-4edb-43a2-9caf-6a5b0b699ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:51.916427 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2e40d-4edb-43a2-9caf-6a5b0b699ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:51.938582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-9ae2e40d-4edb-43a2-9caf-6a5b0b699ea5 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:51.940342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9ae2e40d-4edb-43a2-9caf-6a5b0b699ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:51.945226 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8cf5c5-ad3c-46d9-b30a-f9ddde4cf862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:51.945836 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cf5c5-ad3c-46d9-b30a-f9ddde4cf862 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:51.946114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cf5c5-ad3c-46d9-b30a-f9ddde4cf862 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:51.946382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cf5c5-ad3c-46d9-b30a-f9ddde4cf862 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:51.954870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:51.958993 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:52.010621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 87/177] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:52.012073 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c21247-4875-42a3-b700-44a190ca5229 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:52.012358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c21247-4875-42a3-b700-44a190ca5229 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:52.012626 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c21247-4875-42a3-b700-44a190ca5229 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:52.013870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 88/178] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:52.018994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce664bd4-931b-4437-90b6-3869d4ed1456 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:52.018994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce664bd4-931b-4437-90b6-3869d4ed1456 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:52.018994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce664bd4-931b-4437-90b6-3869d4ed1456 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:52.018994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 89/179] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:52.018994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce664bd4-931b-4437-90b6-3869d4ed1456 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:52.023782 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8cf5c5-ad3c-46d9-b30a-f9ddde4cf862 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:52.025258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8cf5c5-ad3c-46d9-b30a-f9ddde4cf862 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:52.129778 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce664bd4-931b-4437-90b6-3869d4ed1456 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:52.141258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 91/180] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:13:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:13:52.142585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022b8d3-1727-4bf1-ac38-bbcecbfa35b7 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:52.142957 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022b8d3-1727-4bf1-ac38-bbcecbfa35b7 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:52.143300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022b8d3-1727-4bf1-ac38-bbcecbfa35b7 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:52.146815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 92/181] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:52.146815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d200fc-9779-4ca3-a493-01e42963524a None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:52.146815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d200fc-9779-4ca3-a493-01e42963524a None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:52.146815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d200fc-9779-4ca3-a493-01e42963524a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:52.164395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-23d200fc-9779-4ca3-a493-01e42963524a None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:52.165340 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-23d200fc-9779-4ca3-a493-01e42963524a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:52.172603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800f357-23d4-49a7-941e-d89243b5b165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:52.173243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800f357-23d4-49a7-941e-d89243b5b165 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:52.173521 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800f357-23d4-49a7-941e-d89243b5b165 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:52.173796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800f357-23d4-49a7-941e-d89243b5b165 None admin] PATH_INFO: `/v3/projects/4a0c087d2d50474e856da494b5d6563b` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:52.174954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800f357-23d4-49a7-941e-d89243b5b165 None admin] RBAC: Authorizing `identity:get_project(project_id=4a0c087d2d50474e856da494b5d6563b)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:52.176411 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800f357-23d4-49a7-941e-d89243b5b165 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:52.178312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 90/182] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:52 2022] GET /identity/v3/projects/4a0c087d2d50474e856da494b5d6563b => generated 312 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:52.179439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb5bf5-5c90-4e89-978f-5db50f95defe None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:52.179719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb5bf5-5c90-4e89-978f-5db50f95defe None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:52.179993 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb5bf5-5c90-4e89-978f-5db50f95defe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:52.198314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-54cb5bf5-5c90-4e89-978f-5db50f95defe None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:52.199534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-54cb5bf5-5c90-4e89-978f-5db50f95defe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:52.199963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:52.215372 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:52.237798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 93/183] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:52.241042 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d200fc-9779-4ca3-a493-01e42963524a None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:52.278910 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 91/184] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:52.280243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a18e7-4fe5-4356-9b51-213ab46c3fa6 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:52.280528 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a18e7-4fe5-4356-9b51-213ab46c3fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:52.280799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a18e7-4fe5-4356-9b51-213ab46c3fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:52.301822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-258a18e7-4fe5-4356-9b51-213ab46c3fa6 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:52.303053 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-258a18e7-4fe5-4356-9b51-213ab46c3fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:52.353124 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d200fc-9779-4ca3-a493-01e42963524a None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:52.374463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 92/185] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:52.375200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a18e7-4fe5-4356-9b51-213ab46c3fa6 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:52.389741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d384f-a7d2-4c30-bd75-d77b06d3998c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:52.390767 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d384f-a7d2-4c30-bd75-d77b06d3998c None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:52.391368 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d384f-a7d2-4c30-bd75-d77b06d3998c None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:52.391846 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d384f-a7d2-4c30-bd75-d77b06d3998c None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:52.393218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d384f-a7d2-4c30-bd75-d77b06d3998c None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:52.394955 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d384f-a7d2-4c30-bd75-d77b06d3998c None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:52.397148 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 94/186] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:52 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:52.398139 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d384f-a7d2-4c30-bd75-d77b06d3998c None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:52.435072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:52.465390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:52.486250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a18e7-4fe5-4356-9b51-213ab46c3fa6 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:52.502641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d384f-a7d2-4c30-bd75-d77b06d3998c None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:52.522611 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac32036-fbf9-4ada-a57f-80217c5b0168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:52.523446 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac32036-fbf9-4ada-a57f-80217c5b0168 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:52.523867 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac32036-fbf9-4ada-a57f-80217c5b0168 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:52.524300 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac32036-fbf9-4ada-a57f-80217c5b0168 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:52.525174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac32036-fbf9-4ada-a57f-80217c5b0168 None admin] RBAC: Adding query filter params (role.id=5812e6538f3f4bdfbfbe11cd25c02957, scope.project.id=4a0c087d2d50474e856da494b5d6563b, user.id=b0cca89fab4740169878ca61aabace33) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:52.526584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac32036-fbf9-4ada-a57f-80217c5b0168 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:52.528539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac32036-fbf9-4ada-a57f-80217c5b0168 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:52.535963 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 93/187] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:13:52 2022] GET /identity/v3/role_assignments?user.id=b0cca89fab4740169878ca61aabace33&scope.project.id=4a0c087d2d50474e856da494b5d6563b&role.id=5812e6538f3f4bdfbfbe11cd25c02957 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:52.537527 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac32036-fbf9-4ada-a57f-80217c5b0168 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:52.542290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4db508-00de-48da-8cf1-564c4ecaaf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:52.542999 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4db508-00de-48da-8cf1-564c4ecaaf8d None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:52.543382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4db508-00de-48da-8cf1-564c4ecaaf8d None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:52.543755 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4db508-00de-48da-8cf1-564c4ecaaf8d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:52.556536 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4db508-00de-48da-8cf1-564c4ecaaf8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:52.562133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-bc4db508-00de-48da-8cf1-564c4ecaaf8d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:13:52.562133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:52.562133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:52.562133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:52.562133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:52.562133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:52.562133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:52.562133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:52.562133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:52.562133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:52.562133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:52.562133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:52.562133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:52.562133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:52.562133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:52.562133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:52.563238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:52.563238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:52.563238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:52.563238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:52.563238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:52.563238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:52.563238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:52.563238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:52.563238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:52.563238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:52.563238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:52.563238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:52.563238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:52.563238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:52.563238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:52.563238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:52.564221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:52.564221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:52.564221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:52.564221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:52.564221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:52.564221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:52.564221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:52.564221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:52.564221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:52.564221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:52.564221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:13:52.564221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:13:52.566213 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 95/188] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:52.571529 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4db508-00de-48da-8cf1-564c4ecaaf8d None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:52.660128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac32036-fbf9-4ada-a57f-80217c5b0168 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:52.701794 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afaaff7-1540-412a-9308-3cae20c63101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:52.703060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaaff7-1540-412a-9308-3cae20c63101 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:52.703725 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaaff7-1540-412a-9308-3cae20c63101 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:52.704158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaaff7-1540-412a-9308-3cae20c63101 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:52.705313 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afaaff7-1540-412a-9308-3cae20c63101 None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=2269c4a7f991405497de468e3a9a745a, user.id=40eedd635c074248bf2c6c67ebf16bb0) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:52.705621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afaaff7-1540-412a-9308-3cae20c63101 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:52.707511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afaaff7-1540-412a-9308-3cae20c63101 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:52.713788 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 94/189] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:13:52 2022] GET /identity/v3/role_assignments?user.id=40eedd635c074248bf2c6c67ebf16bb0&scope.project.id=2269c4a7f991405497de468e3a9a745a&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:52.719680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afaaff7-1540-412a-9308-3cae20c63101 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:52.740918 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:52.762732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:52.767294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4db508-00de-48da-8cf1-564c4ecaaf8d None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:52.816378 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d7289-9890-431b-bb7c-eab97f9c2d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:52.817095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d7289-9890-431b-bb7c-eab97f9c2d6e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:52.817378 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d7289-9890-431b-bb7c-eab97f9c2d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:52.817652 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d7289-9890-431b-bb7c-eab97f9c2d6e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:52.830149 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d7289-9890-431b-bb7c-eab97f9c2d6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:52.836877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-fd3d7289-9890-431b-bb7c-eab97f9c2d6e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:13:52.836877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:52.836877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:52.836877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:52.836877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:52.836877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:52.836877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:52.836877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:52.836877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:52.836877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:52.836877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:52.836877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:52.836877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:52.836877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:52.836877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:52.836877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:52.837842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:52.837842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:52.837842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:52.837842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:52.837842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:52.837842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:52.837842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:52.837842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:52.837842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:52.837842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:52.837842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:52.837842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:52.837842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:52.837842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:52.837842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:52.837842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:52.839237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:52.839237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:52.839237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:52.839237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:52.839237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:52.839237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:52.839237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:52.839237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:52.839237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:52.839237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:52.839237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:13:52.839237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:13:52.839237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 96/190] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:13:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:52.839919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d7289-9890-431b-bb7c-eab97f9c2d6e None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:52.861910 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afaaff7-1540-412a-9308-3cae20c63101 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:52.904475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33483164-a9cf-41f3-9a0a-819927507da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:52.905091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33483164-a9cf-41f3-9a0a-819927507da1 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:52.905374 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33483164-a9cf-41f3-9a0a-819927507da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:52.905650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33483164-a9cf-41f3-9a0a-819927507da1 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:52.906314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33483164-a9cf-41f3-9a0a-819927507da1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:52.906613 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33483164-a9cf-41f3-9a0a-819927507da1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:52.908032 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33483164-a9cf-41f3-9a0a-819927507da1 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:52.916679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 95/191] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:13:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:52.917318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33483164-a9cf-41f3-9a0a-819927507da1 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:52.949526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d7289-9890-431b-bb7c-eab97f9c2d6e None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:52.985008 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ef477-3606-48c3-8895-12e4affaf606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:52.985644 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ef477-3606-48c3-8895-12e4affaf606 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:52.985928 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ef477-3606-48c3-8895-12e4affaf606 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:52.986205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ef477-3606-48c3-8895-12e4affaf606 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:52.990177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:52.996178 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ef477-3606-48c3-8895-12e4affaf606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:53.001385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-816ef477-3606-48c3-8895-12e4affaf606 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:13:53.001385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:53.001385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:53.001385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:53.001385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:53.001385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:53.001385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:53.001385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:53.001385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:53.001385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:53.001385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:53.001385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:53.001385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:53.001385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:53.001385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:53.001385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:53.004951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:53.004951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:53.004951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:53.004951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:53.004951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:53.004951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:53.004951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:53.004951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:53.004951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:53.004951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:53.004951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:53.004951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:53.004951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:53.004951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:53.004951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:53.004951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:53.005774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:53.005774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:53.005774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:53.005774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:53.005774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:53.005774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:53.005774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:53.005774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:53.005774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:53.005774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:53.005774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:13:53.005774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:13:53.005774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 97/192] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:13:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:53.005774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ef477-3606-48c3-8895-12e4affaf606 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:53.021512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:53.022707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33483164-a9cf-41f3-9a0a-819927507da1 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:53.056532 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23fbb4-31a4-406f-ae90-0dd0b2934723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:53.057202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23fbb4-31a4-406f-ae90-0dd0b2934723 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:53.057489 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23fbb4-31a4-406f-ae90-0dd0b2934723 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:53.057766 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23fbb4-31a4-406f-ae90-0dd0b2934723 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:53.058912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23fbb4-31a4-406f-ae90-0dd0b2934723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:53.060367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23fbb4-31a4-406f-ae90-0dd0b2934723 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:53.061925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 96/193] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:13:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:53.062680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23fbb4-31a4-406f-ae90-0dd0b2934723 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:53.106360 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ef477-3606-48c3-8895-12e4affaf606 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:53.140840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08272292-3b01-4527-8774-c9cf6a662602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:53.141467 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08272292-3b01-4527-8774-c9cf6a662602 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:53.141744 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08272292-3b01-4527-8774-c9cf6a662602 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:53.142014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08272292-3b01-4527-8774-c9cf6a662602 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:53.142669 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08272292-3b01-4527-8774-c9cf6a662602 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:53.142998 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08272292-3b01-4527-8774-c9cf6a662602 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:53.144375 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08272292-3b01-4527-8774-c9cf6a662602 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:53.152090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 98/194] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:13:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:53.152795 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08272292-3b01-4527-8774-c9cf6a662602 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:53.165616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23fbb4-31a4-406f-ae90-0dd0b2934723 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:53.199511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012d850-7b7f-4894-aa8b-a0bb3ddc8f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:53.200144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012d850-7b7f-4894-aa8b-a0bb3ddc8f33 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:53.200424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012d850-7b7f-4894-aa8b-a0bb3ddc8f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:53.200704 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012d850-7b7f-4894-aa8b-a0bb3ddc8f33 None admin] PATH_INFO: `/v3/users/0bb1f8cff11d4177879e801bed0e8db4` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:53.201817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012d850-7b7f-4894-aa8b-a0bb3ddc8f33 None admin] RBAC: Authorizing `identity:get_user(user_id=0bb1f8cff11d4177879e801bed0e8db4)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:53.203292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012d850-7b7f-4894-aa8b-a0bb3ddc8f33 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:53.204854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 97/195] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:53 2022] GET /identity/v3/users/0bb1f8cff11d4177879e801bed0e8db4 => generated 289 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:53.205578 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012d850-7b7f-4894-aa8b-a0bb3ddc8f33 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:53.214707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:53.233545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:53.261532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08272292-3b01-4527-8774-c9cf6a662602 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:53.285804 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2071dc1-d452-4572-a373-3157873e685f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:53.286409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2071dc1-d452-4572-a373-3157873e685f None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:53.286689 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2071dc1-d452-4572-a373-3157873e685f None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:53.286986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2071dc1-d452-4572-a373-3157873e685f None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:53.287642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2071dc1-d452-4572-a373-3157873e685f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:53.287943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2071dc1-d452-4572-a373-3157873e685f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:53.289306 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2071dc1-d452-4572-a373-3157873e685f None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:53.297262 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 99/196] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:53.302007 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2071dc1-d452-4572-a373-3157873e685f None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:53.302684 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012d850-7b7f-4894-aa8b-a0bb3ddc8f33 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:53.335815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e16727-1f33-4e4d-8feb-6b8bb8b99102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:53.336510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e16727-1f33-4e4d-8feb-6b8bb8b99102 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:53.336787 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e16727-1f33-4e4d-8feb-6b8bb8b99102 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:53.337055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e16727-1f33-4e4d-8feb-6b8bb8b99102 None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:53.337689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e16727-1f33-4e4d-8feb-6b8bb8b99102 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:53.339216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e16727-1f33-4e4d-8feb-6b8bb8b99102 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:53.398295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2071dc1-d452-4572-a373-3157873e685f None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:53.429561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a172e-7274-4017-800b-3a73e57fce1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:53.430173 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a172e-7274-4017-800b-3a73e57fce1a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:53.430450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a172e-7274-4017-800b-3a73e57fce1a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:53.430721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a172e-7274-4017-800b-3a73e57fce1a None admin] PATH_INFO: `/v3/users/f50eb207ae494a8a819a20a3bf503965` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:53.441327 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:53.444148 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a172e-7274-4017-800b-3a73e57fce1a None admin] RBAC: Authorizing `identity:get_user(user_id=f50eb207ae494a8a819a20a3bf503965)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:53.445590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a172e-7274-4017-800b-3a73e57fce1a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:53.447165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 100/197] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:53 2022] GET /identity/v3/users/f50eb207ae494a8a819a20a3bf503965 => generated 283 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:53.448566 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a172e-7274-4017-800b-3a73e57fce1a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:53.473149 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 98/198] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:13:53 2022] POST /identity/v3/users => generated 297 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:13:53.480242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e16727-1f33-4e4d-8feb-6b8bb8b99102 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:53.549481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:53.561164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a172e-7274-4017-800b-3a73e57fce1a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:53.592125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e16727-1f33-4e4d-8feb-6b8bb8b99102 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:53.598201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b954d5-3aa9-4e2f-8078-da7001ff54d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:53.598852 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b954d5-3aa9-4e2f-8078-da7001ff54d2 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:53.599134 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b954d5-3aa9-4e2f-8078-da7001ff54d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:53.599404 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b954d5-3aa9-4e2f-8078-da7001ff54d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:53.632851 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15d15f-0f9a-433a-8fdd-7702f403c828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:53.633918 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15d15f-0f9a-433a-8fdd-7702f403c828 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:53.634253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15d15f-0f9a-433a-8fdd-7702f403c828 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:53.634534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15d15f-0f9a-433a-8fdd-7702f403c828 None admin] PATH_INFO: `/v3/users/f50eb207ae494a8a819a20a3bf503965` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:53.635776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15d15f-0f9a-433a-8fdd-7702f403c828 None admin] RBAC: Authorizing `identity:get_user(user_id=f50eb207ae494a8a819a20a3bf503965)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:53.637228 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15d15f-0f9a-433a-8fdd-7702f403c828 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:53.638828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 99/199] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:53 2022] GET /identity/v3/users/f50eb207ae494a8a819a20a3bf503965 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:53.639563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15d15f-0f9a-433a-8fdd-7702f403c828 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:53.676917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:53.680950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b954d5-3aa9-4e2f-8078-da7001ff54d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:53.683153 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b954d5-3aa9-4e2f-8078-da7001ff54d2 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:53.768132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15d15f-0f9a-433a-8fdd-7702f403c828 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:53.803206 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:53.808994 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 101/200] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:13:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:13:53.809866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b954d5-3aa9-4e2f-8078-da7001ff54d2 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:53.811319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53015e2e-6329-42f3-b503-4e1073c924f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:53.812494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53015e2e-6329-42f3-b503-4e1073c924f3 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:53.815589 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53015e2e-6329-42f3-b503-4e1073c924f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:53.815589 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53015e2e-6329-42f3-b503-4e1073c924f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:53.895893 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53015e2e-6329-42f3-b503-4e1073c924f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:53.897513 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53015e2e-6329-42f3-b503-4e1073c924f3 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:53.923472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:53.928201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b954d5-3aa9-4e2f-8078-da7001ff54d2 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:53.965569 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7370a-4846-4046-af0f-261bc4af7b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:53.966543 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7370a-4846-4046-af0f-261bc4af7b97 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:53.967110 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7370a-4846-4046-af0f-261bc4af7b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:53.967596 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7370a-4846-4046-af0f-261bc4af7b97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:54.001453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 100/201] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:13:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:13:54.002311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53015e2e-6329-42f3-b503-4e1073c924f3 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:54.039233 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:54.039803 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7370a-4846-4046-af0f-261bc4af7b97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:54.041598 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7370a-4846-4046-af0f-261bc4af7b97 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:54.112570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53015e2e-6329-42f3-b503-4e1073c924f3 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:54.151000 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:54.151520 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163da51-56f8-497a-8ef7-9ea459b68ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:54.152604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163da51-56f8-497a-8ef7-9ea459b68ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:54.152991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163da51-56f8-497a-8ef7-9ea459b68ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:54.153368 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163da51-56f8-497a-8ef7-9ea459b68ce6 None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:54.154613 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163da51-56f8-497a-8ef7-9ea459b68ce6 None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:54.156204 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163da51-56f8-497a-8ef7-9ea459b68ce6 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:54.157713 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 102/202] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:13:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:13:54.158388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 101/203] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:54 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:54.159085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7370a-4846-4046-af0f-261bc4af7b97 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:54.164096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163da51-56f8-497a-8ef7-9ea459b68ce6 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:54.261656 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7370a-4846-4046-af0f-261bc4af7b97 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:54.271952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163da51-56f8-497a-8ef7-9ea459b68ce6 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:54.275079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:54.297219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa30f6-4eb0-4ef5-a990-141be61120d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:54.298189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa30f6-4eb0-4ef5-a990-141be61120d3 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:54.298658 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa30f6-4eb0-4ef5-a990-141be61120d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:54.299151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa30f6-4eb0-4ef5-a990-141be61120d3 None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:54.300460 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa30f6-4eb0-4ef5-a990-141be61120d3 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:54.302095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa30f6-4eb0-4ef5-a990-141be61120d3 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:54.303954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 103/204] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:54 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:54.304978 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa30f6-4eb0-4ef5-a990-141be61120d3 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:54.313273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe1fa2-de1b-48c0-bc7f-36ec459ff640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:54.313892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe1fa2-de1b-48c0-bc7f-36ec459ff640 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:54.314235 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe1fa2-de1b-48c0-bc7f-36ec459ff640 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:54.314516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe1fa2-de1b-48c0-bc7f-36ec459ff640 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:54.315896 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe1fa2-de1b-48c0-bc7f-36ec459ff640 None admin] RBAC: Adding query filter params (role.id=9c013d602e9640b1883700191dcbd10a, scope.project.id=2269c4a7f991405497de468e3a9a745a, user.id=0bb1f8cff11d4177879e801bed0e8db4) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:54.316038 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe1fa2-de1b-48c0-bc7f-36ec459ff640 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:54.317478 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe1fa2-de1b-48c0-bc7f-36ec459ff640 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:54.331636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 102/205] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:13:54 2022] GET /identity/v3/role_assignments?user.id=0bb1f8cff11d4177879e801bed0e8db4&scope.project.id=2269c4a7f991405497de468e3a9a745a&role.id=9c013d602e9640b1883700191dcbd10a => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:13:54.332607 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe1fa2-de1b-48c0-bc7f-36ec459ff640 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:54.377810 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:54.456806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa30f6-4eb0-4ef5-a990-141be61120d3 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:54.481957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe1fa2-de1b-48c0-bc7f-36ec459ff640 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:54.536105 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d10c4b-0c5b-4f8e-bdfc-af4c38122cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:54.536759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d10c4b-0c5b-4f8e-bdfc-af4c38122cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:54.537042 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d10c4b-0c5b-4f8e-bdfc-af4c38122cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:54.537318 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d10c4b-0c5b-4f8e-bdfc-af4c38122cf0 None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:54.539159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d10c4b-0c5b-4f8e-bdfc-af4c38122cf0 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:54.540612 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d10c4b-0c5b-4f8e-bdfc-af4c38122cf0 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:54.543020 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 104/206] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:54 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:54.550478 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d10c4b-0c5b-4f8e-bdfc-af4c38122cf0 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:54.552323 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252bd5d-c2b4-42ac-8f46-555a3a2816c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:54.553170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252bd5d-c2b4-42ac-8f46-555a3a2816c2 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:54.553793 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252bd5d-c2b4-42ac-8f46-555a3a2816c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:54.554285 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252bd5d-c2b4-42ac-8f46-555a3a2816c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:54.555754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252bd5d-c2b4-42ac-8f46-555a3a2816c2 None admin] RBAC: Adding query filter params (role.id=1182ffc3e9f9477db2003500b7bfd402, scope.project.id=e5e3e090ce8348c19e0ddc4994094302, user.id=f50eb207ae494a8a819a20a3bf503965) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:54.556076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252bd5d-c2b4-42ac-8f46-555a3a2816c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:54.557586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252bd5d-c2b4-42ac-8f46-555a3a2816c2 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:54.569592 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 103/207] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:13:54 2022] GET /identity/v3/role_assignments?user.id=f50eb207ae494a8a819a20a3bf503965&scope.project.id=e5e3e090ce8348c19e0ddc4994094302&role.id=1182ffc3e9f9477db2003500b7bfd402 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:54.687347 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d10c4b-0c5b-4f8e-bdfc-af4c38122cf0 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:54.735881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfcf39-e21b-41e9-83a0-732ea6392860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:54.736963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfcf39-e21b-41e9-83a0-732ea6392860 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:54.737447 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfcf39-e21b-41e9-83a0-732ea6392860 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:54.737937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfcf39-e21b-41e9-83a0-732ea6392860 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:54.739042 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cfcf39-e21b-41e9-83a0-732ea6392860 None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=e5e3e090ce8348c19e0ddc4994094302, user.id=f50eb207ae494a8a819a20a3bf503965) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:54.739556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cfcf39-e21b-41e9-83a0-732ea6392860 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:54.741377 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cfcf39-e21b-41e9-83a0-732ea6392860 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:54.743844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:54.752867 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 105/208] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:13:54 2022] GET /identity/v3/role_assignments?user.id=f50eb207ae494a8a819a20a3bf503965&scope.project.id=e5e3e090ce8348c19e0ddc4994094302&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:13:55.600600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7aec04-9815-4b5b-8c5e-4d731ae7f292 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:55.601121 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7aec04-9815-4b5b-8c5e-4d731ae7f292 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:55.601195 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7aec04-9815-4b5b-8c5e-4d731ae7f292 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:55.602739 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 104/209] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:55.608174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e515bd-93ac-42e5-9075-c12bb64c1d16 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:55.608591 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e515bd-93ac-42e5-9075-c12bb64c1d16 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:55.608763 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e515bd-93ac-42e5-9075-c12bb64c1d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:55.622282 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef65d4e-8f63-4b1f-a478-dd5d5e4a3eb6 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:55.622585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef65d4e-8f63-4b1f-a478-dd5d5e4a3eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:55.622857 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef65d4e-8f63-4b1f-a478-dd5d5e4a3eb6 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:55.624457 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 105/210] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:55.629523 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c78155-2d00-40f6-b96d-77c1dba9a369 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:55.630040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c78155-2d00-40f6-b96d-77c1dba9a369 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:55.630414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-87e515bd-93ac-42e5-9075-c12bb64c1d16 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:55.630748 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c78155-2d00-40f6-b96d-77c1dba9a369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:55.631667 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-87e515bd-93ac-42e5-9075-c12bb64c1d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:55.650970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-f2c78155-2d00-40f6-b96d-77c1dba9a369 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:55.652476 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f2c78155-2d00-40f6-b96d-77c1dba9a369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:55.679294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:55.705295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 106/211] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:55.739516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 106/212] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:13:55.977627 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e515bd-93ac-42e5-9075-c12bb64c1d16 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:56.018591 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c78155-2d00-40f6-b96d-77c1dba9a369 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:56.107126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e515bd-93ac-42e5-9075-c12bb64c1d16 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:56.138010 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:56.146398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a73073-8498-4667-a6d1-48d6aad3363b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:56.146867 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c78155-2d00-40f6-b96d-77c1dba9a369 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:56.147570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a73073-8498-4667-a6d1-48d6aad3363b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:56.148138 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a73073-8498-4667-a6d1-48d6aad3363b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:56.148614 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a73073-8498-4667-a6d1-48d6aad3363b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:56.160551 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a73073-8498-4667-a6d1-48d6aad3363b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:56.166112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-61a73073-8498-4667-a6d1-48d6aad3363b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:13:56.166112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:56.166112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:56.166112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:56.166112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:56.166112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:56.166112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:56.166112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:56.166112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:56.166112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:56.166112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:56.166112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:56.166112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:56.166112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:56.166112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:56.166112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:56.167251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:56.167251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:56.167251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:56.167251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:56.167251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:56.167251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:56.167251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:56.167251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:56.167251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:56.167251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:56.167251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:56.167251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:56.167251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:56.167251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:56.167251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:56.167251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:56.168127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:56.168127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:56.168127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:56.168127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:56.168127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:56.168127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:56.168127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:56.168127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:56.168127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:56.168127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:56.168127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:13:56.168127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:13:56.175235 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 107/213] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:56.176461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a73073-8498-4667-a6d1-48d6aad3363b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:56.185860 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c955ac4-cb26-4d8d-b588-2504c1c33870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:56.186573 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c955ac4-cb26-4d8d-b588-2504c1c33870 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:56.186903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c955ac4-cb26-4d8d-b588-2504c1c33870 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:56.187179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c955ac4-cb26-4d8d-b588-2504c1c33870 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:56.188986 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c955ac4-cb26-4d8d-b588-2504c1c33870 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:56.191426 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c955ac4-cb26-4d8d-b588-2504c1c33870 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:56.198010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 107/214] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:13:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:13:56.199513 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c955ac4-cb26-4d8d-b588-2504c1c33870 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:56.243335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:56.301650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a73073-8498-4667-a6d1-48d6aad3363b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:56.318858 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c955ac4-cb26-4d8d-b588-2504c1c33870 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:56.345001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc80f52-4312-4f69-a9ca-bbd84e81ca26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:56.345969 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc80f52-4312-4f69-a9ca-bbd84e81ca26 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:56.346450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc80f52-4312-4f69-a9ca-bbd84e81ca26 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:56.346974 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc80f52-4312-4f69-a9ca-bbd84e81ca26 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:56.347866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc80f52-4312-4f69-a9ca-bbd84e81ca26 None admin] RBAC: Adding query filter params (name=member) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:56.348389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc80f52-4312-4f69-a9ca-bbd84e81ca26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:56.350068 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc80f52-4312-4f69-a9ca-bbd84e81ca26 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:56.359770 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 108/215] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:13:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:56.364522 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb96d2-411f-40d3-bf43-5a094b671957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:56.364522 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb96d2-411f-40d3-bf43-5a094b671957 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:56.364522 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb96d2-411f-40d3-bf43-5a094b671957 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:56.364522 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb96d2-411f-40d3-bf43-5a094b671957 None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:56.364900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb96d2-411f-40d3-bf43-5a094b671957 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:56.365930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc80f52-4312-4f69-a9ca-bbd84e81ca26 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:56.366316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb96d2-411f-40d3-bf43-5a094b671957 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:56.421026 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:56.457478 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 108/216] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:13:56 2022] POST /identity/v3/users => generated 297 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:13:56.480059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc80f52-4312-4f69-a9ca-bbd84e81ca26 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:56.520408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9e93a-3bcd-45d0-9890-47b0824ddeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:56.521158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9e93a-3bcd-45d0-9890-47b0824ddeca None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:56.521431 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9e93a-3bcd-45d0-9890-47b0824ddeca None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:56.521707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9e93a-3bcd-45d0-9890-47b0824ddeca None admin] PATH_INFO: `/v3/users/e3f37b568c5b4d4fb65ae6518ddc8fc4` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:56.537546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9e93a-3bcd-45d0-9890-47b0824ddeca None admin] RBAC: Authorizing `identity:get_user(user_id=e3f37b568c5b4d4fb65ae6518ddc8fc4)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:56.539159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9e93a-3bcd-45d0-9890-47b0824ddeca None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:56.541716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 109/217] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:56 2022] GET /identity/v3/users/e3f37b568c5b4d4fb65ae6518ddc8fc4 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:56.547876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb96d2-411f-40d3-bf43-5a094b671957 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:56.574560 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99a02b-8c70-457a-8114-973762296dd7 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:56.574912 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99a02b-8c70-457a-8114-973762296dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:56.575166 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99a02b-8c70-457a-8114-973762296dd7 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:56.576815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 110/218] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:56.582243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920da46-53dc-4552-97da-a6a2e28514e9 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:56.582523 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920da46-53dc-4552-97da-a6a2e28514e9 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:56.582816 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920da46-53dc-4552-97da-a6a2e28514e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:56.602811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-1920da46-53dc-4552-97da-a6a2e28514e9 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:56.604495 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-1920da46-53dc-4552-97da-a6a2e28514e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:56.653731 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:56.675843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb96d2-411f-40d3-bf43-5a094b671957 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:56.688657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 111/219] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:13:56.690296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f166d0-80f1-4dce-84de-79af3c5da24e None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:56.690586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f166d0-80f1-4dce-84de-79af3c5da24e None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:56.690884 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f166d0-80f1-4dce-84de-79af3c5da24e None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:56.694887 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 112/220] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:56.697954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9451147-2d5a-4b6a-8aee-6091540e4c85 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:56.700092 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9451147-2d5a-4b6a-8aee-6091540e4c85 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:56.700092 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9451147-2d5a-4b6a-8aee-6091540e4c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:56.718046 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-d9451147-2d5a-4b6a-8aee-6091540e4c85 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:56.719274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d9451147-2d5a-4b6a-8aee-6091540e4c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:56.721925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec63c98-f31c-40ac-9653-11ba02f54715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:56.723498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec63c98-f31c-40ac-9653-11ba02f54715 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:56.723849 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec63c98-f31c-40ac-9653-11ba02f54715 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:56.724192 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec63c98-f31c-40ac-9653-11ba02f54715 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:56.758175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:56.806945 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec63c98-f31c-40ac-9653-11ba02f54715 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:56.807115 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 113/221] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:13:56.808486 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec63c98-f31c-40ac-9653-11ba02f54715 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:56.897061 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:56.916883 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 109/222] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:13:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:13:56.922877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9451147-2d5a-4b6a-8aee-6091540e4c85 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:56.964260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec63c98-f31c-40ac-9653-11ba02f54715 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:57.042810 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9451147-2d5a-4b6a-8aee-6091540e4c85 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:57.096081 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91624643-6906-43bf-a488-7bb93e477308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:57.096201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91624643-6906-43bf-a488-7bb93e477308 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:57.100217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91624643-6906-43bf-a488-7bb93e477308 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:57.100217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91624643-6906-43bf-a488-7bb93e477308 None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:57.100217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91624643-6906-43bf-a488-7bb93e477308 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:57.100217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91624643-6906-43bf-a488-7bb93e477308 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:57.101877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 114/223] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:56 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:13:57.107570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91624643-6906-43bf-a488-7bb93e477308 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:57.123226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec63c98-f31c-40ac-9653-11ba02f54715 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:57.152329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:57.165937 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c153ff-54e9-4b7c-a89b-bd05cae1e286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:57.168032 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c153ff-54e9-4b7c-a89b-bd05cae1e286 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:57.168032 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c153ff-54e9-4b7c-a89b-bd05cae1e286 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:57.168032 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c153ff-54e9-4b7c-a89b-bd05cae1e286 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:57.184917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c153ff-54e9-4b7c-a89b-bd05cae1e286 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:57.191358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-33c153ff-54e9-4b7c-a89b-bd05cae1e286 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:13:57.191358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:57.191358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:57.191358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:57.191358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:57.191358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:57.191358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:57.191358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:57.191358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:57.191358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:57.191358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:57.191358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:57.191358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:57.191358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:57.191358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:57.191358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:57.192289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:57.192289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:57.192289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:57.192289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:57.192289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:57.192289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:57.192289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:57.192289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:57.192289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:57.192289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:57.192289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:57.192289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:57.192289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:57.192289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:57.192289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:57.192289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:57.193108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:57.193108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:57.193108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:57.193108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:57.193108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:57.193108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:57.193108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:57.193108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:57.193108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:57.193108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:57.193108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:13:57.193108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:13:57.193108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 110/224] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:13:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:13:57.193807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c153ff-54e9-4b7c-a89b-bd05cae1e286 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:57.241067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91624643-6906-43bf-a488-7bb93e477308 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:57.242068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:57.289408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376dd96-84c9-4983-9276-0deaa7c28ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:57.290339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376dd96-84c9-4983-9276-0deaa7c28ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:57.290769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376dd96-84c9-4983-9276-0deaa7c28ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:57.291742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376dd96-84c9-4983-9276-0deaa7c28ed9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:57.305544 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c153ff-54e9-4b7c-a89b-bd05cae1e286 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:57.306525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376dd96-84c9-4983-9276-0deaa7c28ed9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:57.312865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-1376dd96-84c9-4983-9276-0deaa7c28ed9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:13:57.312865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:57.312865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:57.312865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:57.312865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:57.312865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:57.312865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:57.312865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:57.312865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:57.312865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:57.312865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:57.312865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:57.312865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:57.312865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:57.312865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:57.312865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:57.313968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:57.313968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:57.313968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:57.313968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:57.313968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:57.313968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:57.313968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:57.313968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:57.313968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:57.313968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:57.313968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:57.313968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:57.313968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:57.313968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:57.313968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:57.313968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:57.315296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:57.315296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:57.315296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:57.315296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:57.315296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:57.315296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:57.315296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:57.315296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:57.315296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:57.315296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:57.315296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:13:57.315296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:13:57.322821 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 115/225] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:13:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:13:57.322821 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376dd96-84c9-4983-9276-0deaa7c28ed9 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:57.354472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbac71-be8f-41b5-9661-31146ce55014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:57.355372 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbac71-be8f-41b5-9661-31146ce55014 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:57.355801 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbac71-be8f-41b5-9661-31146ce55014 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:57.356218 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbac71-be8f-41b5-9661-31146ce55014 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:57.357154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbac71-be8f-41b5-9661-31146ce55014 None admin] RBAC: Adding query filter params (role.id=5812e6538f3f4bdfbfbe11cd25c02957, scope.project.id=e5e3e090ce8348c19e0ddc4994094302, user.id=e3f37b568c5b4d4fb65ae6518ddc8fc4) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:57.357612 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbac71-be8f-41b5-9661-31146ce55014 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:57.359386 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbac71-be8f-41b5-9661-31146ce55014 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:57.371192 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 111/226] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:13:57 2022] GET /identity/v3/role_assignments?user.id=e3f37b568c5b4d4fb65ae6518ddc8fc4&scope.project.id=e5e3e090ce8348c19e0ddc4994094302&role.id=5812e6538f3f4bdfbfbe11cd25c02957 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:57.372354 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbac71-be8f-41b5-9661-31146ce55014 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:57.464634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:57.478386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376dd96-84c9-4983-9276-0deaa7c28ed9 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:57.536805 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbac71-be8f-41b5-9661-31146ce55014 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:57.558815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:57.567229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc14ae-32d5-4e3d-9552-2f18650ef000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:57.567865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc14ae-32d5-4e3d-9552-2f18650ef000 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:57.568145 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc14ae-32d5-4e3d-9552-2f18650ef000 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:57.573037 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc14ae-32d5-4e3d-9552-2f18650ef000 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:57.573037 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc14ae-32d5-4e3d-9552-2f18650ef000 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:57.573037 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc14ae-32d5-4e3d-9552-2f18650ef000 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:57.573037 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc14ae-32d5-4e3d-9552-2f18650ef000 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:57.585837 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 116/227] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:13:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:57.591367 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc14ae-32d5-4e3d-9552-2f18650ef000 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:57.630752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2b2e0-3a47-4131-ad20-57fbcdae0bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:57.631333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2b2e0-3a47-4131-ad20-57fbcdae0bbd None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:57.631619 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2b2e0-3a47-4131-ad20-57fbcdae0bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:57.631897 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2b2e0-3a47-4131-ad20-57fbcdae0bbd None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:57.632574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2b2e0-3a47-4131-ad20-57fbcdae0bbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:13:57.632880 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2b2e0-3a47-4131-ad20-57fbcdae0bbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:57.634270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2b2e0-3a47-4131-ad20-57fbcdae0bbd None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:57.664810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 112/228] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:57.671761 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2b2e0-3a47-4131-ad20-57fbcdae0bbd None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:57.699408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc14ae-32d5-4e3d-9552-2f18650ef000 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:57.736169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb2953-00de-4f0e-821c-8ca68bb1ac49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:57.736816 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb2953-00de-4f0e-821c-8ca68bb1ac49 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:57.737096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb2953-00de-4f0e-821c-8ca68bb1ac49 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:57.737372 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb2953-00de-4f0e-821c-8ca68bb1ac49 None admin] PATH_INFO: `/v3/users/f50eb207ae494a8a819a20a3bf503965` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:57.738512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb2953-00de-4f0e-821c-8ca68bb1ac49 None admin] RBAC: Authorizing `identity:get_user(user_id=f50eb207ae494a8a819a20a3bf503965)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:57.739996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb2953-00de-4f0e-821c-8ca68bb1ac49 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:57.742170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 117/229] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:57 2022] GET /identity/v3/users/f50eb207ae494a8a819a20a3bf503965 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:57.748237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb2953-00de-4f0e-821c-8ca68bb1ac49 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:57.787181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:57.889007 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb2953-00de-4f0e-821c-8ca68bb1ac49 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:57.904475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2b2e0-3a47-4131-ad20-57fbcdae0bbd None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:57.942196 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a068a8-701a-4d44-add2-d4ace43e39d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:57.943232 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a068a8-701a-4d44-add2-d4ace43e39d3 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:57.943780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a068a8-701a-4d44-add2-d4ace43e39d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:57.944323 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a068a8-701a-4d44-add2-d4ace43e39d3 None admin] PATH_INFO: `/v3/users/f50eb207ae494a8a819a20a3bf503965` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:57.945717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a068a8-701a-4d44-add2-d4ace43e39d3 None admin] RBAC: Authorizing `identity:get_user(user_id=f50eb207ae494a8a819a20a3bf503965)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:57.947954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a068a8-701a-4d44-add2-d4ace43e39d3 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:57.950062 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 113/230] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:13:57 2022] GET /identity/v3/users/f50eb207ae494a8a819a20a3bf503965 => generated 283 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:57.955822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a068a8-701a-4d44-add2-d4ace43e39d3 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:57.973199 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff797e-c63b-4e19-8730-1bc20ae1dcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:57.973624 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff797e-c63b-4e19-8730-1bc20ae1dcdc None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:57.973907 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff797e-c63b-4e19-8730-1bc20ae1dcdc None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:57.974185 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff797e-c63b-4e19-8730-1bc20ae1dcdc None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:57.975399 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff797e-c63b-4e19-8730-1bc20ae1dcdc None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:57.976813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff797e-c63b-4e19-8730-1bc20ae1dcdc None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:57.987355 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 118/231] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:57 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:13:57.992561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff797e-c63b-4e19-8730-1bc20ae1dcdc None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:57.995590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:58.069975 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a068a8-701a-4d44-add2-d4ace43e39d3 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:58.110333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8692d-e5b2-456b-bf63-7f24b6bf4e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:58.111307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8692d-e5b2-456b-bf63-7f24b6bf4e79 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:58.111811 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8692d-e5b2-456b-bf63-7f24b6bf4e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:58.112288 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8692d-e5b2-456b-bf63-7f24b6bf4e79 None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:58.113615 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e8692d-e5b2-456b-bf63-7f24b6bf4e79 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:58.115319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e8692d-e5b2-456b-bf63-7f24b6bf4e79 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:58.117696 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 114/232] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:13:57 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:13:58.126707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8692d-e5b2-456b-bf63-7f24b6bf4e79 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:58.160799 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:58.173425 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff797e-c63b-4e19-8730-1bc20ae1dcdc None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:58.234752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8692d-e5b2-456b-bf63-7f24b6bf4e79 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:58.243091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:58.276121 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ac68f-0169-4d95-bb69-348a04bd9335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:58.276918 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ac68f-0169-4d95-bb69-348a04bd9335 None admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:58.277308 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ac68f-0169-4d95-bb69-348a04bd9335 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:58.277717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ac68f-0169-4d95-bb69-348a04bd9335 None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302/users/f50eb207ae494a8a819a20a3bf503965/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:58.279820 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ac68f-0169-4d95-bb69-348a04bd9335 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5e3e090ce8348c19e0ddc4994094302, user_id=f50eb207ae494a8a819a20a3bf503965, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:58.281597 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ac68f-0169-4d95-bb69-348a04bd9335 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:58.287514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd43f07-e4d9-4080-8f98-bffc6677324e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:58.288380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd43f07-e4d9-4080-8f98-bffc6677324e None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:58.288767 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd43f07-e4d9-4080-8f98-bffc6677324e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:58.289162 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd43f07-e4d9-4080-8f98-bffc6677324e None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302/users/f50eb207ae494a8a819a20a3bf503965/roles/1182ffc3e9f9477db2003500b7bfd402` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:58.292072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd43f07-e4d9-4080-8f98-bffc6677324e None admin] RBAC: Authorizing `identity:create_grant(project_id=e5e3e090ce8348c19e0ddc4994094302, user_id=f50eb207ae494a8a819a20a3bf503965, role_id=1182ffc3e9f9477db2003500b7bfd402)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:58.293697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd43f07-e4d9-4080-8f98-bffc6677324e None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:58.365340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 115/233] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:13:58 2022] PUT /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302/users/f50eb207ae494a8a819a20a3bf503965/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:13:58.400272 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 119/234] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:13:57 2022] PUT /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302/users/f50eb207ae494a8a819a20a3bf503965/roles/1182ffc3e9f9477db2003500b7bfd402 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:13:59.076380 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7dd78-687f-4e71-92f1-f11161090547 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:59.076916 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7dd78-687f-4e71-92f1-f11161090547 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:59.076916 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7dd78-687f-4e71-92f1-f11161090547 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:59.078512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 116/235] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:59.084092 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f151bb8-0ff7-4e1d-9b80-9609a3d5fcb4 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:59.084517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f151bb8-0ff7-4e1d-9b80-9609a3d5fcb4 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:59.084698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f151bb8-0ff7-4e1d-9b80-9609a3d5fcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:59.104390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-3f151bb8-0ff7-4e1d-9b80-9609a3d5fcb4 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:59.105590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-3f151bb8-0ff7-4e1d-9b80-9609a3d5fcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:59.132495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62275a-665c-4e02-9b10-bd9d4e012f04 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:59.132815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62275a-665c-4e02-9b10-bd9d4e012f04 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:59.133051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62275a-665c-4e02-9b10-bd9d4e012f04 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:59.134679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 117/236] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:59.152540 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b262e0-d03a-4a3d-bbf3-c834ac0c0496 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:59.152829 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b262e0-d03a-4a3d-bbf3-c834ac0c0496 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:59.153096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b262e0-d03a-4a3d-bbf3-c834ac0c0496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:59.183564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-33b262e0-d03a-4a3d-bbf3-c834ac0c0496 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:59.184769 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-33b262e0-d03a-4a3d-bbf3-c834ac0c0496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:59.210254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 120/237] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:13:59.313016 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 118/238] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:59.608129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f151bb8-0ff7-4e1d-9b80-9609a3d5fcb4 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:59.625791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b262e0-d03a-4a3d-bbf3-c834ac0c0496 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:59.768859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:59.786738 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f151bb8-0ff7-4e1d-9b80-9609a3d5fcb4 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:59.812961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:13:59.821195 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b262e0-d03a-4a3d-bbf3-c834ac0c0496 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:59.831309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015f249-4b23-418a-bec3-05838bb14895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:59.836454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015f249-4b23-418a-bec3-05838bb14895 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:59.836454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015f249-4b23-418a-bec3-05838bb14895 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:59.836454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015f249-4b23-418a-bec3-05838bb14895 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:59.836454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015f249-4b23-418a-bec3-05838bb14895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:13:59.836454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015f249-4b23-418a-bec3-05838bb14895 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:13:59.838490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 121/239] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:13:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:13:59.842835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bcdd7-12d1-4db6-8886-06f6a45424ee None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:59.842835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bcdd7-12d1-4db6-8886-06f6a45424ee None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:59.842835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bcdd7-12d1-4db6-8886-06f6a45424ee None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:59.842835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 122/240] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:13:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:13:59.846430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29bcdd7-12d1-4db6-8886-06f6a45424ee None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:13:59.867070 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d6f91-14d6-4d8c-9fc0-b2f96618f090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:13:59.867984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d6f91-14d6-4d8c-9fc0-b2f96618f090 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:59.868464 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d6f91-14d6-4d8c-9fc0-b2f96618f090 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:59.869061 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d6f91-14d6-4d8c-9fc0-b2f96618f090 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:59.881422 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d6f91-14d6-4d8c-9fc0-b2f96618f090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:13:59.887011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-577d6f91-14d6-4d8c-9fc0-b2f96618f090 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:13:59.887011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:13:59.887011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:13:59.887011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:13:59.887011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:13:59.887011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:13:59.887011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:13:59.887011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:13:59.887011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:13:59.887011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:13:59.887011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:13:59.887011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:13:59.887011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:13:59.887011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:13:59.887011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:13:59.887011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:13:59.887886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:13:59.887886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:13:59.887886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:13:59.887886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:13:59.887886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:13:59.887886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:13:59.887886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:13:59.887886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:13:59.887886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:13:59.887886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:13:59.887886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:13:59.887886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:13:59.887886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:13:59.887886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:13:59.887886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:13:59.887886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:13:59.888671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:13:59.888671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:13:59.888671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:13:59.888671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:13:59.888671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:13:59.888671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:13:59.888671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:13:59.888671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:13:59.888671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:13:59.888671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:13:59.888671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:13:59.888671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:13:59.896110 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 119/241] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:13:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:13:59.898329 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad41a46-c929-4fc3-92f2-91b2f1c4ce78 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:13:59.898878 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad41a46-c929-4fc3-92f2-91b2f1c4ce78 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:13:59.899340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad41a46-c929-4fc3-92f2-91b2f1c4ce78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:13:59.918614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-5ad41a46-c929-4fc3-92f2-91b2f1c4ce78 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:13:59.920327 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-5ad41a46-c929-4fc3-92f2-91b2f1c4ce78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:13:59.969366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29bcdd7-12d1-4db6-8886-06f6a45424ee None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:13:59.991487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 120/242] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:13:59.992652 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad41a46-c929-4fc3-92f2-91b2f1c4ce78 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:00.034864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b77bb-d8f0-4f12-a9df-ef62536555c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:00.035812 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b77bb-d8f0-4f12-a9df-ef62536555c5 None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:00.036270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b77bb-d8f0-4f12-a9df-ef62536555c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:00.036729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b77bb-d8f0-4f12-a9df-ef62536555c5 None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:00.037830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b77bb-d8f0-4f12-a9df-ef62536555c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:00.039483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b77bb-d8f0-4f12-a9df-ef62536555c5 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:00.132459 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:00.148705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 123/243] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:13:59 2022] POST /identity/v3/users => generated 293 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:14:00.169834 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad41a46-c929-4fc3-92f2-91b2f1c4ce78 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:00.217448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa1496-0fc1-4755-a3bf-c26bdfe14298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:00.218375 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa1496-0fc1-4755-a3bf-c26bdfe14298 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:00.218895 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa1496-0fc1-4755-a3bf-c26bdfe14298 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:00.219393 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa1496-0fc1-4755-a3bf-c26bdfe14298 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:00.220366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa1496-0fc1-4755-a3bf-c26bdfe14298 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:00.220845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa1496-0fc1-4755-a3bf-c26bdfe14298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:00.222412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa1496-0fc1-4755-a3bf-c26bdfe14298 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:00.232172 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 121/244] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:13:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:00.238425 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b77bb-d8f0-4f12-a9df-ef62536555c5 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:00.251756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:00.361882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b77bb-d8f0-4f12-a9df-ef62536555c5 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:00.369835 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa1496-0fc1-4755-a3bf-c26bdfe14298 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:00.413626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b5ae5-e725-4ab5-bd9f-9e2eb96f0051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:00.414202 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b5ae5-e725-4ab5-bd9f-9e2eb96f0051 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:00.414489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b5ae5-e725-4ab5-bd9f-9e2eb96f0051 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:00.414768 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b5ae5-e725-4ab5-bd9f-9e2eb96f0051 None admin] PATH_INFO: `/v3/users/7d54389fbedc4d75ba9c8fdae74db9db` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:00.430366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b5ae5-e725-4ab5-bd9f-9e2eb96f0051 None admin] RBAC: Authorizing `identity:get_user(user_id=7d54389fbedc4d75ba9c8fdae74db9db)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:00.432585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b5ae5-e725-4ab5-bd9f-9e2eb96f0051 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:00.434550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 124/245] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:14:00 2022] GET /identity/v3/users/7d54389fbedc4d75ba9c8fdae74db9db => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:00.440792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b5ae5-e725-4ab5-bd9f-9e2eb96f0051 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:00.483618 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa1496-0fc1-4755-a3bf-c26bdfe14298 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:00.518880 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4293252-6058-4c3c-abc7-a9d1c67b6c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:00.519523 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4293252-6058-4c3c-abc7-a9d1c67b6c8c None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:00.519808 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4293252-6058-4c3c-abc7-a9d1c67b6c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:00.520085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4293252-6058-4c3c-abc7-a9d1c67b6c8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:00.528995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:00.532196 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4293252-6058-4c3c-abc7-a9d1c67b6c8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:00.538250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-b4293252-6058-4c3c-abc7-a9d1c67b6c8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:00.538250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:00.538250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:00.538250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:00.538250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:00.538250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:00.538250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:00.538250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:00.538250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:00.538250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:00.538250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:00.538250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:00.538250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:00.538250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:00.538250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:00.538250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:00.544705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:00.544705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:00.544705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:00.544705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:00.544705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:00.544705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:00.544705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:00.544705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:00.544705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:00.544705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:00.544705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:00.544705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:00.544705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:00.544705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:00.544705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:00.544705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:00.545782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:00.545782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:00.545782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:00.545782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:00.545782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:00.545782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:00.545782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:00.545782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:00.545782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:00.545782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:00.545782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:00.545782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:00.545782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 122/246] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:00.545782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdec710-57aa-4594-bbbe-6bcd7a79f87c None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:00.546655 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdec710-57aa-4594-bbbe-6bcd7a79f87c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:00.546655 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdec710-57aa-4594-bbbe-6bcd7a79f87c None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:00.546655 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 123/247] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:00.554966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1311ae8-cdc4-4ad9-b3b5-7b881ec2e054 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:00.555336 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1311ae8-cdc4-4ad9-b3b5-7b881ec2e054 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:00.555687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1311ae8-cdc4-4ad9-b3b5-7b881ec2e054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:00.569883 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:00.576519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b5ae5-e725-4ab5-bd9f-9e2eb96f0051 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:00.577991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-e1311ae8-cdc4-4ad9-b3b5-7b881ec2e054 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:00.579312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e1311ae8-cdc4-4ad9-b3b5-7b881ec2e054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:00.621972 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674d83e-2ca2-4b83-a1f3-70b175f179ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:00.622780 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674d83e-2ca2-4b83-a1f3-70b175f179ef None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:00.623098 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674d83e-2ca2-4b83-a1f3-70b175f179ef None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:00.623539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674d83e-2ca2-4b83-a1f3-70b175f179ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:00.678417 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 124/248] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:00.679213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1311ae8-cdc4-4ad9-b3b5-7b881ec2e054 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:00.718981 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674d83e-2ca2-4b83-a1f3-70b175f179ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:00.720581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674d83e-2ca2-4b83-a1f3-70b175f179ef None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:00.801485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1311ae8-cdc4-4ad9-b3b5-7b881ec2e054 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:00.814921 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:00.873508 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19018a-f1bb-4f99-882c-47bd4cc842fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:00.874183 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19018a-f1bb-4f99-882c-47bd4cc842fa None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:00.874465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19018a-f1bb-4f99-882c-47bd4cc842fa None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:00.874732 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19018a-f1bb-4f99-882c-47bd4cc842fa None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:00.877263 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19018a-f1bb-4f99-882c-47bd4cc842fa None admin] RBAC: Adding query filter params (name=member) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:00.877567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19018a-f1bb-4f99-882c-47bd4cc842fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:00.882105 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19018a-f1bb-4f99-882c-47bd4cc842fa None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:00.886018 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 125/249] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 14:14:00.887165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b80f-3431-42e2-9e55-23152c50bbb7 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:00.887412 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b80f-3431-42e2-9e55-23152c50bbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:00.887684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b80f-3431-42e2-9e55-23152c50bbb7 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:00.890331 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:00.897319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 126/250] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:00 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:00.901583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901bbd7-0876-4503-b31c-9c3a80d1cdf1 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:00.901583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901bbd7-0876-4503-b31c-9c3a80d1cdf1 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:00.901583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901bbd7-0876-4503-b31c-9c3a80d1cdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:00.903041 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 125/251] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:14:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:00.903883 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19018a-f1bb-4f99-882c-47bd4cc842fa None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:00.933061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-7901bbd7-0876-4503-b31c-9c3a80d1cdf1 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:00.938823 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7901bbd7-0876-4503-b31c-9c3a80d1cdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:01.012851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 127/252] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:01.014195 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901bbd7-0876-4503-b31c-9c3a80d1cdf1 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:01.067303 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19018a-f1bb-4f99-882c-47bd4cc842fa None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:01.111431 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584ae87-e26a-41c2-a0a1-5e7808800dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:01.112067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584ae87-e26a-41c2-a0a1-5e7808800dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:01.112347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584ae87-e26a-41c2-a0a1-5e7808800dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:01.112617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584ae87-e26a-41c2-a0a1-5e7808800dc7 None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:01.113756 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584ae87-e26a-41c2-a0a1-5e7808800dc7 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:01.115347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584ae87-e26a-41c2-a0a1-5e7808800dc7 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:01.117448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 126/253] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:00 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:01.118480 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584ae87-e26a-41c2-a0a1-5e7808800dc7 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:01.144864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:01.153514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901bbd7-0876-4503-b31c-9c3a80d1cdf1 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:01.192535 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd05c6e-b99e-4f89-b9c8-f6a928aa4186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:01.193394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd05c6e-b99e-4f89-b9c8-f6a928aa4186 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:01.193812 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd05c6e-b99e-4f89-b9c8-f6a928aa4186 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:01.194251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd05c6e-b99e-4f89-b9c8-f6a928aa4186 None admin] PATH_INFO: `/v3/users/e3f37b568c5b4d4fb65ae6518ddc8fc4` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:01.195654 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd05c6e-b99e-4f89-b9c8-f6a928aa4186 None admin] RBAC: Authorizing `identity:get_user(user_id=e3f37b568c5b4d4fb65ae6518ddc8fc4)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:01.197865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd05c6e-b99e-4f89-b9c8-f6a928aa4186 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:01.199687 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 128/254] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:14:01 2022] GET /identity/v3/users/e3f37b568c5b4d4fb65ae6518ddc8fc4 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:01.200641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd05c6e-b99e-4f89-b9c8-f6a928aa4186 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:01.209504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:01.258543 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584ae87-e26a-41c2-a0a1-5e7808800dc7 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:01.297914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84889f-e773-4e72-97ca-43d914caf250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:01.298560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84889f-e773-4e72-97ca-43d914caf250 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:01.298865 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84889f-e773-4e72-97ca-43d914caf250 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:01.299148 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84889f-e773-4e72-97ca-43d914caf250 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:01.310882 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84889f-e773-4e72-97ca-43d914caf250 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:01.318306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-ed84889f-e773-4e72-97ca-43d914caf250 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:14:01.318306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:01.318306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:01.318306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:01.318306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:01.318306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:01.318306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:01.318306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:01.318306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:01.318306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:01.318306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:01.318306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:01.318306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:01.318306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:01.318306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:01.318306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:01.319199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:01.319199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:01.319199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:01.319199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:01.319199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:01.319199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:01.319199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:01.319199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:01.319199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:01.319199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:01.319199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:01.319199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:01.319199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:01.319199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:01.319199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:01.319199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:01.319992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:01.319992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:01.319992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:01.319992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:01.319992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:01.319992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:01.319992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:01.319992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:01.319992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:01.319992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:01.319992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:14:01.319992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:01.319992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 127/255] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:14:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:01.320644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84889f-e773-4e72-97ca-43d914caf250 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:01.373595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd05c6e-b99e-4f89-b9c8-f6a928aa4186 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:01.410425 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98173d4-7c84-46b6-aba6-21e4695e2a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:01.411414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98173d4-7c84-46b6-aba6-21e4695e2a52 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:01.411891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98173d4-7c84-46b6-aba6-21e4695e2a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:01.412388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98173d4-7c84-46b6-aba6-21e4695e2a52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:01.413303 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98173d4-7c84-46b6-aba6-21e4695e2a52 None admin] RBAC: Adding query filter params (role.id=9c013d602e9640b1883700191dcbd10a, scope.project.id=e5e3e090ce8348c19e0ddc4994094302, user.id=7d54389fbedc4d75ba9c8fdae74db9db) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:01.413823 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98173d4-7c84-46b6-aba6-21e4695e2a52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:01.415466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98173d4-7c84-46b6-aba6-21e4695e2a52 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:01.421789 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 129/256] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:14:01 2022] GET /identity/v3/role_assignments?user.id=7d54389fbedc4d75ba9c8fdae74db9db&scope.project.id=e5e3e090ce8348c19e0ddc4994094302&role.id=9c013d602e9640b1883700191dcbd10a => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:01.423243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98173d4-7c84-46b6-aba6-21e4695e2a52 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:01.432168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84889f-e773-4e72-97ca-43d914caf250 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:01.455415 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:01.465941 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:01.497933 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef844c3-d88c-4ec4-bcd1-02896a1b94ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:01.498680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef844c3-d88c-4ec4-bcd1-02896a1b94ff None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:01.499088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef844c3-d88c-4ec4-bcd1-02896a1b94ff None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:01.499433 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef844c3-d88c-4ec4-bcd1-02896a1b94ff None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:01.503307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef844c3-d88c-4ec4-bcd1-02896a1b94ff None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:01.504866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef844c3-d88c-4ec4-bcd1-02896a1b94ff None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:01.507039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 128/257] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:01 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:01.507717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef844c3-d88c-4ec4-bcd1-02896a1b94ff None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:01.548638 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98173d4-7c84-46b6-aba6-21e4695e2a52 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:01.591523 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7e55e-df79-4333-837d-424ecbd6fd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:01.592544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7e55e-df79-4333-837d-424ecbd6fd9f None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:01.593018 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7e55e-df79-4333-837d-424ecbd6fd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:01.593479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7e55e-df79-4333-837d-424ecbd6fd9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:01.605147 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7e55e-df79-4333-837d-424ecbd6fd9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:01.611129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-80e7e55e-df79-4333-837d-424ecbd6fd9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:14:01.611129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:01.611129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:01.611129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:01.611129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:01.611129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:01.611129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:01.611129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:01.611129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:01.611129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:01.611129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:01.611129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:01.611129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:01.611129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:01.611129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:01.611129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:01.612002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:01.612002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:01.612002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:01.612002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:01.612002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:01.612002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:01.612002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:01.612002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:01.612002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:01.612002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:01.612002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:01.612002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:01.612002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:01.612002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:01.612002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:01.612002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:01.612798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:01.612798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:01.612798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:01.612798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:01.612798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:01.612798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:01.612798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:01.612798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:01.612798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:01.612798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:01.612798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:14:01.612798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:01.615554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 130/258] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:14:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:01.616644 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7e55e-df79-4333-837d-424ecbd6fd9f None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:01.663393 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef844c3-d88c-4ec4-bcd1-02896a1b94ff None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:01.700671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a58b6-bd9b-43cd-bcb2-de335cd1bdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:01.701278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a58b6-bd9b-43cd-bcb2-de335cd1bdc1 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:01.701555 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a58b6-bd9b-43cd-bcb2-de335cd1bdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:01.701825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a58b6-bd9b-43cd-bcb2-de335cd1bdc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:01.702483 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a58b6-bd9b-43cd-bcb2-de335cd1bdc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:01.702780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a58b6-bd9b-43cd-bcb2-de335cd1bdc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:01.704301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a58b6-bd9b-43cd-bcb2-de335cd1bdc1 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:01.719443 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:01.726542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 129/259] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:01.727713 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a58b6-bd9b-43cd-bcb2-de335cd1bdc1 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:01.738192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7e55e-df79-4333-837d-424ecbd6fd9f None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:01.803290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34cfb2-7abc-426f-b22d-cc5db047e016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:01.808339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34cfb2-7abc-426f-b22d-cc5db047e016 None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:01.808625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34cfb2-7abc-426f-b22d-cc5db047e016 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:01.808903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34cfb2-7abc-426f-b22d-cc5db047e016 None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302/users/e3f37b568c5b4d4fb65ae6518ddc8fc4/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:01.810945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34cfb2-7abc-426f-b22d-cc5db047e016 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5e3e090ce8348c19e0ddc4994094302, user_id=e3f37b568c5b4d4fb65ae6518ddc8fc4, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:01.812743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34cfb2-7abc-426f-b22d-cc5db047e016 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:01.833955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:01.886771 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 131/260] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:14:01 2022] PUT /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302/users/e3f37b568c5b4d4fb65ae6518ddc8fc4/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:14:01.887683 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34cfb2-7abc-426f-b22d-cc5db047e016 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:01.895563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a58b6-bd9b-43cd-bcb2-de335cd1bdc1 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:01.994600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbf4e2-2a92-4ad0-8ab3-2dbd49513259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:01.995149 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbf4e2-2a92-4ad0-8ab3-2dbd49513259 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:01.995431 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbf4e2-2a92-4ad0-8ab3-2dbd49513259 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:01.995700 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbf4e2-2a92-4ad0-8ab3-2dbd49513259 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:01.996366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbf4e2-2a92-4ad0-8ab3-2dbd49513259 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:01.996893 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbf4e2-2a92-4ad0-8ab3-2dbd49513259 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:01.998292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbf4e2-2a92-4ad0-8ab3-2dbd49513259 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:02.016288 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 130/261] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:02.021578 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbf4e2-2a92-4ad0-8ab3-2dbd49513259 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:02.114044 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34cfb2-7abc-426f-b22d-cc5db047e016 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:02.205570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42233683-0310-4e81-8f48-b828dd08bbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:02.206337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42233683-0310-4e81-8f48-b828dd08bbb6 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:02.206501 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42233683-0310-4e81-8f48-b828dd08bbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:02.206765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42233683-0310-4e81-8f48-b828dd08bbb6 None admin] PATH_INFO: `/v3/users/f50eb207ae494a8a819a20a3bf503965` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:02.208200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42233683-0310-4e81-8f48-b828dd08bbb6 None admin] RBAC: Authorizing `identity:get_user(user_id=f50eb207ae494a8a819a20a3bf503965)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:02.210601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42233683-0310-4e81-8f48-b828dd08bbb6 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:02.212330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 132/262] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:14:01 2022] GET /identity/v3/users/f50eb207ae494a8a819a20a3bf503965 => generated 283 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:02.218737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42233683-0310-4e81-8f48-b828dd08bbb6 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:02.255921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbf4e2-2a92-4ad0-8ab3-2dbd49513259 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:02.266614 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:02.278366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:02.299262 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cd9d6-5cc3-47c6-8b9d-d07cb5eb33cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:02.299961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cd9d6-5cc3-47c6-8b9d-d07cb5eb33cb None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:02.300237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cd9d6-5cc3-47c6-8b9d-d07cb5eb33cb None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:02.300721 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cd9d6-5cc3-47c6-8b9d-d07cb5eb33cb None admin] PATH_INFO: `/v3/users/f50eb207ae494a8a819a20a3bf503965` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:02.301875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cd9d6-5cc3-47c6-8b9d-d07cb5eb33cb None admin] RBAC: Authorizing `identity:get_user(user_id=f50eb207ae494a8a819a20a3bf503965)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:02.304920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cd9d6-5cc3-47c6-8b9d-d07cb5eb33cb None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:02.307388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 131/263] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:14:02 2022] GET /identity/v3/users/f50eb207ae494a8a819a20a3bf503965 => generated 283 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:02.318645 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cd9d6-5cc3-47c6-8b9d-d07cb5eb33cb None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:02.370575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42233683-0310-4e81-8f48-b828dd08bbb6 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:02.425214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe1ae7-9cd2-40ff-8b6b-5220cd1dfa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:02.426331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe1ae7-9cd2-40ff-8b6b-5220cd1dfa20 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:02.426835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe1ae7-9cd2-40ff-8b6b-5220cd1dfa20 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:02.427340 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe1ae7-9cd2-40ff-8b6b-5220cd1dfa20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:02.495361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cd9d6-5cc3-47c6-8b9d-d07cb5eb33cb None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:02.534384 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe1ae7-9cd2-40ff-8b6b-5220cd1dfa20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:02.536218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe1ae7-9cd2-40ff-8b6b-5220cd1dfa20 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:02.575150 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:02.575509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3908c-887d-4202-8798-c31184e7cb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:02.576344 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3908c-887d-4202-8798-c31184e7cb60 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:02.576453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3908c-887d-4202-8798-c31184e7cb60 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:02.576678 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3908c-887d-4202-8798-c31184e7cb60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:02.681597 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:02.696836 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3908c-887d-4202-8798-c31184e7cb60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:02.698858 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3908c-887d-4202-8798-c31184e7cb60 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:02.789871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 133/264] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 14:14:02.795917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe1ae7-9cd2-40ff-8b6b-5220cd1dfa20 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:02.853825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 132/265] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:02.859533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3908c-887d-4202-8798-c31184e7cb60 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:03.003450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe1ae7-9cd2-40ff-8b6b-5220cd1dfa20 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:03.015868 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:03.023612 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:03.037516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3908c-887d-4202-8798-c31184e7cb60 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:03.043134 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84c4ce-7622-4731-bf28-87ad996692d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:03.043828 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84c4ce-7622-4731-bf28-87ad996692d8 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:03.044206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84c4ce-7622-4731-bf28-87ad996692d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:03.044577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84c4ce-7622-4731-bf28-87ad996692d8 None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:03.045803 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84c4ce-7622-4731-bf28-87ad996692d8 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:03.047373 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84c4ce-7622-4731-bf28-87ad996692d8 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:03.049632 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 134/266] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:02 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:03.050875 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed6158-50ee-4cea-a551-63fdf51792c1 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:03.051253 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed6158-50ee-4cea-a551-63fdf51792c1 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:03.051657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed6158-50ee-4cea-a551-63fdf51792c1 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:03.053069 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 135/267] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:03.054911 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55667f-38e3-4db0-b86c-d0d4e302c555 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:03.055247 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55667f-38e3-4db0-b86c-d0d4e302c555 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:03.059429 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55667f-38e3-4db0-b86c-d0d4e302c555 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:03.059429 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 136/268] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:03.064431 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa752d-6151-4dc9-b487-fb0060e02862 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:03.064965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa752d-6151-4dc9-b487-fb0060e02862 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:03.065244 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa752d-6151-4dc9-b487-fb0060e02862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:03.083448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1a023-1680-4ddc-bf47-04e108802f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:03.084091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1a023-1680-4ddc-bf47-04e108802f4b None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:03.084375 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1a023-1680-4ddc-bf47-04e108802f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:03.084649 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1a023-1680-4ddc-bf47-04e108802f4b None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:03.084812 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-adaa752d-6151-4dc9-b487-fb0060e02862 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:03.085778 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1a023-1680-4ddc-bf47-04e108802f4b None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:03.085973 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-adaa752d-6151-4dc9-b487-fb0060e02862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:03.087278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1a023-1680-4ddc-bf47-04e108802f4b None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:03.091028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 133/269] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:02 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:03.092200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4823c5-fa88-4589-99ab-2df941cbf958 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:03.092485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4823c5-fa88-4589-99ab-2df941cbf958 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:03.092763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4823c5-fa88-4589-99ab-2df941cbf958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:03.115050 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-9b4823c5-fa88-4589-99ab-2df941cbf958 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:03.116523 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9b4823c5-fa88-4589-99ab-2df941cbf958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:03.166953 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 137/270] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:03.168352 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa752d-6151-4dc9-b487-fb0060e02862 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:03.198354 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 134/271] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:03.204302 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4823c5-fa88-4589-99ab-2df941cbf958 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:03.320826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:03.341546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa752d-6151-4dc9-b487-fb0060e02862 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:03.348524 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4823c5-fa88-4589-99ab-2df941cbf958 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:03.372838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:03.413168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db433e3-dbdd-457b-915c-07a5335f1f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:03.413787 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db433e3-dbdd-457b-915c-07a5335f1f48 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:03.414068 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db433e3-dbdd-457b-915c-07a5335f1f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:03.414342 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db433e3-dbdd-457b-915c-07a5335f1f48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:03.415139 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db433e3-dbdd-457b-915c-07a5335f1f48 None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=e5e3e090ce8348c19e0ddc4994094302, user.id=f50eb207ae494a8a819a20a3bf503965) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:03.415451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db433e3-dbdd-457b-915c-07a5335f1f48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:03.417454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db433e3-dbdd-457b-915c-07a5335f1f48 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:03.420927 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec4a22-01b2-4117-a3bc-5940f434bcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:03.422126 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec4a22-01b2-4117-a3bc-5940f434bcf6 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:03.422728 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec4a22-01b2-4117-a3bc-5940f434bcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:03.423221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec4a22-01b2-4117-a3bc-5940f434bcf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:03.423896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 138/272] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:14:03 2022] GET /identity/v3/role_assignments?user.id=f50eb207ae494a8a819a20a3bf503965&scope.project.id=e5e3e090ce8348c19e0ddc4994094302&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:03.424371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec4a22-01b2-4117-a3bc-5940f434bcf6 None admin] RBAC: Adding query filter params (role.id=1182ffc3e9f9477db2003500b7bfd402, scope.project.id=e5e3e090ce8348c19e0ddc4994094302, user.id=f50eb207ae494a8a819a20a3bf503965) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:03.424869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec4a22-01b2-4117-a3bc-5940f434bcf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:03.426555 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec4a22-01b2-4117-a3bc-5940f434bcf6 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:03.438090 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 135/273] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:14:03 2022] GET /identity/v3/role_assignments?user.id=f50eb207ae494a8a819a20a3bf503965&scope.project.id=e5e3e090ce8348c19e0ddc4994094302&role.id=1182ffc3e9f9477db2003500b7bfd402 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:03.579100 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db433e3-dbdd-457b-915c-07a5335f1f48 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:03.639545 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec4a22-01b2-4117-a3bc-5940f434bcf6 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:03.707498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db433e3-dbdd-457b-915c-07a5335f1f48 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:03.756809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6347a-e6a5-4558-b279-f589bf16140a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:03.757735 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6347a-e6a5-4558-b279-f589bf16140a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:03.758022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6347a-e6a5-4558-b279-f589bf16140a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:03.758300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6347a-e6a5-4558-b279-f589bf16140a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:03.759781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab6347a-e6a5-4558-b279-f589bf16140a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:03.761703 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab6347a-e6a5-4558-b279-f589bf16140a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:03.766931 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 139/274] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:03.769858 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6347a-e6a5-4558-b279-f589bf16140a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:03.774187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec4a22-01b2-4117-a3bc-5940f434bcf6 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:03.827252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ddd7c-2275-4ab9-9f23-4220e8fb2573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:03.827903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ddd7c-2275-4ab9-9f23-4220e8fb2573 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:03.828183 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ddd7c-2275-4ab9-9f23-4220e8fb2573 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:03.828455 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ddd7c-2275-4ab9-9f23-4220e8fb2573 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:03.843119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ddd7c-2275-4ab9-9f23-4220e8fb2573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:03.845487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:03.848796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-790ddd7c-2275-4ab9-9f23-4220e8fb2573 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:03.848796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:03.848796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:03.848796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:03.848796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:03.848796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:03.848796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:03.848796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:03.848796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:03.848796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:03.848796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:03.848796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:03.848796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:03.848796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:03.848796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:03.848796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:03.849680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:03.849680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:03.849680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:03.849680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:03.849680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:03.849680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:03.849680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:03.849680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:03.849680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:03.849680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:03.849680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:03.849680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:03.849680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:03.849680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:03.849680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:03.849680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:03.850900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:03.850900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:03.850900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:03.850900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:03.850900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:03.850900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:03.850900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:03.850900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:03.850900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:03.850900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:03.850900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:03.850900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:03.850900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 136/275] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:14:03.856261 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ddd7c-2275-4ab9-9f23-4220e8fb2573 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:03.866772 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:03.917727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6347a-e6a5-4558-b279-f589bf16140a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:03.974714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49146f41-93e6-4e02-87f4-aafad6bb4dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:03.975939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49146f41-93e6-4e02-87f4-aafad6bb4dcb None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:03.976844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49146f41-93e6-4e02-87f4-aafad6bb4dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:03.977329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49146f41-93e6-4e02-87f4-aafad6bb4dcb None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:03.978466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49146f41-93e6-4e02-87f4-aafad6bb4dcb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:03.981306 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49146f41-93e6-4e02-87f4-aafad6bb4dcb None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:03.991333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ddd7c-2275-4ab9-9f23-4220e8fb2573 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:04.031727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46f4b3-5e76-4f6a-b24c-3ddc75aa35af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:04.032358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46f4b3-5e76-4f6a-b24c-3ddc75aa35af None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:04.032640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46f4b3-5e76-4f6a-b24c-3ddc75aa35af None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:04.032918 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46f4b3-5e76-4f6a-b24c-3ddc75aa35af None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:04.033586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b46f4b3-5e76-4f6a-b24c-3ddc75aa35af None admin] RBAC: Adding query filter params (name=member) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:04.034049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b46f4b3-5e76-4f6a-b24c-3ddc75aa35af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:04.035572 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b46f4b3-5e76-4f6a-b24c-3ddc75aa35af None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:04.044864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 137/276] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:14:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:04.050279 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46f4b3-5e76-4f6a-b24c-3ddc75aa35af None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:04.108308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 140/277] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:14:03 2022] POST /identity/v3/users => generated 293 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:14:04.114919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d78949-9f20-4393-8fb2-376ec824362c None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:04.114919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d78949-9f20-4393-8fb2-376ec824362c None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:04.114919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d78949-9f20-4393-8fb2-376ec824362c None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:04.114919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 141/278] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:04.117215 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b4931-99a8-4c0e-8956-58ea64cbedc8 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:04.117503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b4931-99a8-4c0e-8956-58ea64cbedc8 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:04.118031 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b4931-99a8-4c0e-8956-58ea64cbedc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:04.133195 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:04.140734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-ed2b4931-99a8-4c0e-8956-58ea64cbedc8 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:04.142023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ed2b4931-99a8-4c0e-8956-58ea64cbedc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:04.174732 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46f4b3-5e76-4f6a-b24c-3ddc75aa35af None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:04.214295 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26e3ca-937f-438c-9f8d-e2180365c6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:04.215379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26e3ca-937f-438c-9f8d-e2180365c6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:04.215856 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26e3ca-937f-438c-9f8d-e2180365c6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:04.216321 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26e3ca-937f-438c-9f8d-e2180365c6f7 None admin] PATH_INFO: `/v3/users/08348ff876fa4490918498ac7e5822cb` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:04.223864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 142/279] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:14:04.234502 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26e3ca-937f-438c-9f8d-e2180365c6f7 None admin] RBAC: Authorizing `identity:get_user(user_id=08348ff876fa4490918498ac7e5822cb)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:04.236436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26e3ca-937f-438c-9f8d-e2180365c6f7 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:04.238827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 138/280] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:14:04 2022] GET /identity/v3/users/08348ff876fa4490918498ac7e5822cb => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:04.249232 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b4931-99a8-4c0e-8956-58ea64cbedc8 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:04.277709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:04.349916 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b4931-99a8-4c0e-8956-58ea64cbedc8 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:04.386643 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad94bc-9323-4e42-baec-5bb7a56bf037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:04.387310 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad94bc-9323-4e42-baec-5bb7a56bf037 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:04.387594 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad94bc-9323-4e42-baec-5bb7a56bf037 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:04.387873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad94bc-9323-4e42-baec-5bb7a56bf037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:04.388601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad94bc-9323-4e42-baec-5bb7a56bf037 None admin] RBAC: Adding query filter params (role.id=5812e6538f3f4bdfbfbe11cd25c02957, user.id=08348ff876fa4490918498ac7e5822cb, scope.system=all) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:04.388913 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad94bc-9323-4e42-baec-5bb7a56bf037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:04.390366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad94bc-9323-4e42-baec-5bb7a56bf037 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:04.396039 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 143/281] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Sep 5 14:14:04 2022] GET /identity/v3/role_assignments?user.id=08348ff876fa4490918498ac7e5822cb&scope.system=all&role.id=5812e6538f3f4bdfbfbe11cd25c02957 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:04.502684 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26e3ca-937f-438c-9f8d-e2180365c6f7 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:04.607873 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26e3ca-937f-438c-9f8d-e2180365c6f7 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:04.634722 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:04.643701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fff649-3ba4-4314-a15e-33a83f306dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:04.644590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fff649-3ba4-4314-a15e-33a83f306dbe None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:04.645027 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fff649-3ba4-4314-a15e-33a83f306dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:04.650763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fff649-3ba4-4314-a15e-33a83f306dbe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:04.656832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fff649-3ba4-4314-a15e-33a83f306dbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:04.661983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-e6fff649-3ba4-4314-a15e-33a83f306dbe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:14:04.661983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:04.661983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:04.661983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:04.661983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:04.661983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:04.661983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:04.661983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:04.661983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:04.661983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:04.661983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:04.661983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:04.661983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:04.661983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:04.661983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:04.661983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:04.662891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:04.662891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:04.662891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:04.662891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:04.662891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:04.662891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:04.662891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:04.662891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:04.662891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:04.662891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:04.662891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:04.662891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:04.662891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:04.662891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:04.662891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:04.662891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:04.663711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:04.663711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:04.663711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:04.663711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:04.663711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:04.663711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:04.663711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:04.663711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:04.663711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:04.663711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:04.663711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:14:04.663711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:04.666891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 139/282] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:04.672691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad94bc-9323-4e42-baec-5bb7a56bf037 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:04.739378 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296445f0-762a-47b0-9f9d-ce39c9267a3c None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:04.740000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296445f0-762a-47b0-9f9d-ce39c9267a3c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:04.740466 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296445f0-762a-47b0-9f9d-ce39c9267a3c None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:04.742810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 140/283] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:04.748370 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238dc2f-a2ff-4557-959e-6d4d4653961e None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:04.748951 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238dc2f-a2ff-4557-959e-6d4d4653961e None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:04.749415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238dc2f-a2ff-4557-959e-6d4d4653961e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:04.773897 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-0238dc2f-a2ff-4557-959e-6d4d4653961e None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:04.778962 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-0238dc2f-a2ff-4557-959e-6d4d4653961e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:04.835163 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:04.838616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad94bc-9323-4e42-baec-5bb7a56bf037 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:04.901849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786e9c1-bb73-4603-899a-bde1263449ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:04.902386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786e9c1-bb73-4603-899a-bde1263449ff None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:04.902664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786e9c1-bb73-4603-899a-bde1263449ff None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:04.902964 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786e9c1-bb73-4603-899a-bde1263449ff None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:04.903634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786e9c1-bb73-4603-899a-bde1263449ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:04.903934 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786e9c1-bb73-4603-899a-bde1263449ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:04.905316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786e9c1-bb73-4603-899a-bde1263449ff None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:04.910280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 141/284] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:04.922609 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 144/285] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:14:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:04.931277 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238dc2f-a2ff-4557-959e-6d4d4653961e None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:05.062178 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238dc2f-a2ff-4557-959e-6d4d4653961e None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:05.084010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:05.114153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62275cb7-7d97-4962-96bd-84a6eb3b23c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:05.114814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62275cb7-7d97-4962-96bd-84a6eb3b23c4 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:05.115151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62275cb7-7d97-4962-96bd-84a6eb3b23c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:05.115431 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62275cb7-7d97-4962-96bd-84a6eb3b23c4 None admin] PATH_INFO: `/v3/users/7d54389fbedc4d75ba9c8fdae74db9db` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:05.119562 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62275cb7-7d97-4962-96bd-84a6eb3b23c4 None admin] RBAC: Authorizing `identity:get_user(user_id=7d54389fbedc4d75ba9c8fdae74db9db)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:05.121006 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62275cb7-7d97-4962-96bd-84a6eb3b23c4 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:05.122863 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 142/286] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:14:04 2022] GET /identity/v3/users/7d54389fbedc4d75ba9c8fdae74db9db => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:05.128497 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786e9c1-bb73-4603-899a-bde1263449ff None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:05.241698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62275cb7-7d97-4962-96bd-84a6eb3b23c4 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:05.279059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786e9c1-bb73-4603-899a-bde1263449ff None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:05.328921 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74483aa2-66f0-46ab-8c6e-ff8f97481e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:05.330644 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74483aa2-66f0-46ab-8c6e-ff8f97481e87 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:05.333597 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74483aa2-66f0-46ab-8c6e-ff8f97481e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:05.334048 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74483aa2-66f0-46ab-8c6e-ff8f97481e87 None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:05.335415 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74483aa2-66f0-46ab-8c6e-ff8f97481e87 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:05.337174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74483aa2-66f0-46ab-8c6e-ff8f97481e87 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:05.349563 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 145/287] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:05 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:05.350601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74483aa2-66f0-46ab-8c6e-ff8f97481e87 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:05.399125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:05.431378 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62275cb7-7d97-4962-96bd-84a6eb3b23c4 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:05.487331 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36734226-526c-4e42-9f95-047fcbb904b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:05.487969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36734226-526c-4e42-9f95-047fcbb904b0 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:05.488248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36734226-526c-4e42-9f95-047fcbb904b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:05.488520 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36734226-526c-4e42-9f95-047fcbb904b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:05.514535 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74483aa2-66f0-46ab-8c6e-ff8f97481e87 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:05.516988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36734226-526c-4e42-9f95-047fcbb904b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:05.523859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-36734226-526c-4e42-9f95-047fcbb904b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:05.523859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:05.523859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:05.523859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:05.523859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:05.523859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:05.523859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:05.523859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:05.523859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:05.523859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:05.523859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:05.523859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:05.523859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:05.523859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:05.523859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:05.523859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:05.524754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:05.524754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:05.524754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:05.524754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:05.524754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:05.524754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:05.524754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:05.524754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:05.524754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:05.524754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:05.524754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:05.524754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:05.524754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:05.524754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:05.524754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:05.524754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:05.525585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:05.525585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:05.525585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:05.525585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:05.525585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:05.525585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:05.525585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:05.525585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:05.525585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:05.525585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:05.525585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:05.525585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:05.531858 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 143/288] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:05.538998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36734226-526c-4e42-9f95-047fcbb904b0 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:05.579922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844ada7-2e4f-43fe-b34e-f4fe52309ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:05.580955 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844ada7-2e4f-43fe-b34e-f4fe52309ebf None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:05.581456 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844ada7-2e4f-43fe-b34e-f4fe52309ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:05.581920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844ada7-2e4f-43fe-b34e-f4fe52309ebf None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302/users/7d54389fbedc4d75ba9c8fdae74db9db/roles/9c013d602e9640b1883700191dcbd10a` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:05.584676 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844ada7-2e4f-43fe-b34e-f4fe52309ebf None admin] RBAC: Authorizing `identity:create_grant(project_id=e5e3e090ce8348c19e0ddc4994094302, user_id=7d54389fbedc4d75ba9c8fdae74db9db, role_id=9c013d602e9640b1883700191dcbd10a)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:05.586731 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844ada7-2e4f-43fe-b34e-f4fe52309ebf None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:05.615253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:05.656324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 146/289] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:14:05 2022] PUT /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302/users/7d54389fbedc4d75ba9c8fdae74db9db/roles/9c013d602e9640b1883700191dcbd10a => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:14:05.685909 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36734226-526c-4e42-9f95-047fcbb904b0 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:05.724477 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a3da1-4695-4d9d-94de-3784804c5a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:05.725442 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a3da1-4695-4d9d-94de-3784804c5a1d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:05.725916 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a3da1-4695-4d9d-94de-3784804c5a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:05.726426 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a3da1-4695-4d9d-94de-3784804c5a1d None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:05.727327 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a3da1-4695-4d9d-94de-3784804c5a1d None admin] RBAC: Adding query filter params (name=member) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:05.727838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a3da1-4695-4d9d-94de-3784804c5a1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:05.730076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a3da1-4695-4d9d-94de-3784804c5a1d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:05.740776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 144/290] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:14:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:05.746705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844ada7-2e4f-43fe-b34e-f4fe52309ebf None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:05.856393 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:05.873587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844ada7-2e4f-43fe-b34e-f4fe52309ebf None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:05.911592 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76c80f-ee13-43ed-bfc3-ed221e962e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:05.912263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76c80f-ee13-43ed-bfc3-ed221e962e25 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:05.912589 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76c80f-ee13-43ed-bfc3-ed221e962e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:05.912865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76c80f-ee13-43ed-bfc3-ed221e962e25 None admin] PATH_INFO: `/v3/users/e3f37b568c5b4d4fb65ae6518ddc8fc4` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:05.914021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76c80f-ee13-43ed-bfc3-ed221e962e25 None admin] RBAC: Authorizing `identity:get_user(user_id=e3f37b568c5b4d4fb65ae6518ddc8fc4)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:05.915502 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76c80f-ee13-43ed-bfc3-ed221e962e25 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:05.917627 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 147/291] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:14:05 2022] GET /identity/v3/users/e3f37b568c5b4d4fb65ae6518ddc8fc4 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:05.923680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a3da1-4695-4d9d-94de-3784804c5a1d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:06.068416 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a3da1-4695-4d9d-94de-3784804c5a1d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:06.076247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:06.105306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ae983-e45f-4ec7-9e18-d3ba2e115257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:06.106027 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ae983-e45f-4ec7-9e18-d3ba2e115257 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:06.106306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ae983-e45f-4ec7-9e18-d3ba2e115257 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:06.106637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ae983-e45f-4ec7-9e18-d3ba2e115257 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:06.179490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ae983-e45f-4ec7-9e18-d3ba2e115257 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:06.180992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ae983-e45f-4ec7-9e18-d3ba2e115257 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:06.307701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 145/292] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:06.312961 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76c80f-ee13-43ed-bfc3-ed221e962e25 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:06.429274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76c80f-ee13-43ed-bfc3-ed221e962e25 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:06.475507 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736f493-dd58-47f8-80a2-080f09cb8a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:06.476520 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736f493-dd58-47f8-80a2-080f09cb8a47 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:06.477008 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736f493-dd58-47f8-80a2-080f09cb8a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:06.477537 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736f493-dd58-47f8-80a2-080f09cb8a47 None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:06.479434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736f493-dd58-47f8-80a2-080f09cb8a47 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:06.481078 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736f493-dd58-47f8-80a2-080f09cb8a47 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:06.482998 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 148/293] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:06 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:06.488813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ae983-e45f-4ec7-9e18-d3ba2e115257 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:06.497620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:06.630328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5996a-bedd-4bda-8fc0-60b66c8f6172 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:06.630998 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5996a-bedd-4bda-8fc0-60b66c8f6172 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:06.631234 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ae983-e45f-4ec7-9e18-d3ba2e115257 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:06.632009 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5996a-bedd-4bda-8fc0-60b66c8f6172 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:06.633825 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 149/294] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:14:06.638964 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778eb5b-2546-4476-ac7a-f2662d6d855a None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:06.640391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778eb5b-2546-4476-ac7a-f2662d6d855a None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:06.640391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778eb5b-2546-4476-ac7a-f2662d6d855a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:06.668844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-f778eb5b-2546-4476-ac7a-f2662d6d855a None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:06.670168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f778eb5b-2546-4476-ac7a-f2662d6d855a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:06.684435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e99e5-3dcb-49a7-a842-a02a5c78dc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:06.684984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e99e5-3dcb-49a7-a842-a02a5c78dc3f None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:06.685370 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e99e5-3dcb-49a7-a842-a02a5c78dc3f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:06.685652 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e99e5-3dcb-49a7-a842-a02a5c78dc3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:06.686389 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e99e5-3dcb-49a7-a842-a02a5c78dc3f None admin] RBAC: Adding query filter params (role.id=5812e6538f3f4bdfbfbe11cd25c02957, scope.project.id=e5e3e090ce8348c19e0ddc4994094302, user.id=e3f37b568c5b4d4fb65ae6518ddc8fc4) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:06.686700 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e99e5-3dcb-49a7-a842-a02a5c78dc3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:06.688173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e99e5-3dcb-49a7-a842-a02a5c78dc3f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:06.694706 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 146/295] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:14:06 2022] GET /identity/v3/role_assignments?user.id=e3f37b568c5b4d4fb65ae6518ddc8fc4&scope.project.id=e5e3e090ce8348c19e0ddc4994094302&role.id=5812e6538f3f4bdfbfbe11cd25c02957 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:06.696008 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fa475-648c-42b2-a2cf-9e740e5bd4e0 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:06.696301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fa475-648c-42b2-a2cf-9e740e5bd4e0 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:06.696578 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fa475-648c-42b2-a2cf-9e740e5bd4e0 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:06.697854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 147/296] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:06.700683 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:06.703671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dd6f4-42fa-499f-b33d-770ad0c3bb1b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:06.703973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dd6f4-42fa-499f-b33d-770ad0c3bb1b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:06.704241 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dd6f4-42fa-499f-b33d-770ad0c3bb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:06.723634 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-b65dd6f4-42fa-499f-b33d-770ad0c3bb1b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:06.724880 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b65dd6f4-42fa-499f-b33d-770ad0c3bb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:06.792401 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:06.803560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 148/298] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:06.803750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 150/298] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:06.951191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05208c73-1b28-431c-a930-a8e5dde24b81 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:06.951479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05208c73-1b28-431c-a930-a8e5dde24b81 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:06.951694 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05208c73-1b28-431c-a930-a8e5dde24b81 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:06.953301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 149/299] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:06.958414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5be50-f548-42a2-81b8-2ac53d50a1f2 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:06.958698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5be50-f548-42a2-81b8-2ac53d50a1f2 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:06.959007 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5be50-f548-42a2-81b8-2ac53d50a1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:06.983190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-80c5be50-f548-42a2-81b8-2ac53d50a1f2 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:06.984396 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-80c5be50-f548-42a2-81b8-2ac53d50a1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:07.057820 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 151/300] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:07.093968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05208c73-1b28-431c-a930-a8e5dde24b81 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:07.136180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5be50-f548-42a2-81b8-2ac53d50a1f2 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:07.224579 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05208c73-1b28-431c-a930-a8e5dde24b81 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:07.235357 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:07.257876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5be50-f548-42a2-81b8-2ac53d50a1f2 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:07.274154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482b706-d3f0-4904-b228-05cec4e69cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:07.275913 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482b706-d3f0-4904-b228-05cec4e69cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:07.277043 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482b706-d3f0-4904-b228-05cec4e69cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:07.278480 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482b706-d3f0-4904-b228-05cec4e69cb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:07.280707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482b706-d3f0-4904-b228-05cec4e69cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:07.282621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482b706-d3f0-4904-b228-05cec4e69cb1 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:07.287975 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 150/301] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:07.295456 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482b706-d3f0-4904-b228-05cec4e69cb1 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:07.305255 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01aa3af-fb07-40eb-b48a-06d55f7669ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:07.305956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aa3af-fb07-40eb-b48a-06d55f7669ad None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:07.306240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aa3af-fb07-40eb-b48a-06d55f7669ad None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:07.306599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aa3af-fb07-40eb-b48a-06d55f7669ad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:07.317750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01aa3af-fb07-40eb-b48a-06d55f7669ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:07.323388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-b01aa3af-fb07-40eb-b48a-06d55f7669ad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:14:07.323388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:07.323388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:07.323388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:07.323388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:07.323388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:07.323388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:07.323388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:07.323388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:07.323388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:07.323388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:07.323388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:07.323388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:07.323388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:07.323388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:07.323388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:07.324270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:07.324270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:07.324270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:07.324270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:07.324270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:07.324270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:07.324270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:07.324270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:07.324270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:07.324270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:07.324270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:07.324270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:07.324270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:07.324270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:07.324270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:07.324270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:07.325072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:07.325072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:07.325072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:07.325072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:07.325072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:07.325072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:07.325072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:07.325072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:07.325072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:07.325072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:07.325072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:14:07.325072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:07.325072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 152/302] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:14:07.330817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01aa3af-fb07-40eb-b48a-06d55f7669ad None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:07.372950 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:07.408036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482b706-d3f0-4904-b228-05cec4e69cb1 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:07.438297 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01aa3af-fb07-40eb-b48a-06d55f7669ad None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:07.444031 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a3656-1cfe-4fc2-9b5d-03f3aa00a83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:07.444649 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a3656-1cfe-4fc2-9b5d-03f3aa00a83a None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:07.444939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a3656-1cfe-4fc2-9b5d-03f3aa00a83a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:07.445218 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a3656-1cfe-4fc2-9b5d-03f3aa00a83a None admin] PATH_INFO: `/v3/groups` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:07.445909 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a3656-1cfe-4fc2-9b5d-03f3aa00a83a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:07.447393 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a3656-1cfe-4fc2-9b5d-03f3aa00a83a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:07.472873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637fa83-3abb-469b-bf71-1cd5b33d96a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:07.473480 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637fa83-3abb-469b-bf71-1cd5b33d96a4 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:07.473760 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637fa83-3abb-469b-bf71-1cd5b33d96a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:07.479104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637fa83-3abb-469b-bf71-1cd5b33d96a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:07.479104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637fa83-3abb-469b-bf71-1cd5b33d96a4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:07.479104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637fa83-3abb-469b-bf71-1cd5b33d96a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:07.479104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637fa83-3abb-469b-bf71-1cd5b33d96a4 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:07.484900 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 153/303] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:14:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:07.485698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637fa83-3abb-469b-bf71-1cd5b33d96a4 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:07.489799 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:07.550820 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 151/304] 10.222.0.15 () {60 vars in 1134 bytes} [Mon Sep 5 14:14:07 2022] POST /identity/v3/groups => generated 229 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:14:07.550820 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a3656-1cfe-4fc2-9b5d-03f3aa00a83a None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:07.610250 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637fa83-3abb-469b-bf71-1cd5b33d96a4 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:07.653029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e56ac7-e6e4-4a31-848f-5504463b740a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:07.653792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e56ac7-e6e4-4a31-848f-5504463b740a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:07.654283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e56ac7-e6e4-4a31-848f-5504463b740a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:07.654633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e56ac7-e6e4-4a31-848f-5504463b740a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:07.668281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e56ac7-e6e4-4a31-848f-5504463b740a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:07.674427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-b3e56ac7-e6e4-4a31-848f-5504463b740a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:07.674427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:07.674427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:07.674427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:07.674427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:07.674427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:07.674427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:07.674427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:07.674427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:07.674427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:07.674427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:07.674427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:07.674427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:07.674427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:07.674427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:07.674427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:07.675424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:07.675424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:07.675424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:07.675424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:07.675424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:07.675424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:07.675424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:07.675424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:07.675424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:07.675424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:07.675424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:07.675424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:07.675424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:07.675424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:07.675424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:07.675424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:07.676295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:07.676295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:07.676295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:07.676295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:07.676295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:07.676295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:07.676295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:07.676295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:07.676295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:07.676295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:07.676295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:07.676295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:07.681071 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 154/305] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:07.682196 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e56ac7-e6e4-4a31-848f-5504463b740a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:07.692646 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a3656-1cfe-4fc2-9b5d-03f3aa00a83a None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:07.729435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:07.739687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5e4ba-97e1-4e04-91e4-5622e2856864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:07.743967 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5e4ba-97e1-4e04-91e4-5622e2856864 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:07.743967 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5e4ba-97e1-4e04-91e4-5622e2856864 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:07.743967 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5e4ba-97e1-4e04-91e4-5622e2856864 None admin] PATH_INFO: `/v3/users/424b8eb6a4f245bbb1a61de4b8ee9397` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:07.758655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:07.762306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5e4ba-97e1-4e04-91e4-5622e2856864 None admin] RBAC: Authorizing `identity:get_user(user_id=424b8eb6a4f245bbb1a61de4b8ee9397)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:07.764533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5e4ba-97e1-4e04-91e4-5622e2856864 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:07.766880 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 152/306] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:14:07 2022] GET /identity/v3/users/424b8eb6a4f245bbb1a61de4b8ee9397 => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:07.771973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5e4ba-97e1-4e04-91e4-5622e2856864 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:07.817539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e56ac7-e6e4-4a31-848f-5504463b740a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:07.862023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd24198-6030-47da-83bf-90b776cf88c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:07.867295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd24198-6030-47da-83bf-90b776cf88c7 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:07.867295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd24198-6030-47da-83bf-90b776cf88c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:07.867295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd24198-6030-47da-83bf-90b776cf88c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:07.867295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd24198-6030-47da-83bf-90b776cf88c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:07.867295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd24198-6030-47da-83bf-90b776cf88c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:07.867295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd24198-6030-47da-83bf-90b776cf88c7 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:07.877061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 155/307] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:14:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:07.882561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd24198-6030-47da-83bf-90b776cf88c7 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:07.890355 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5e4ba-97e1-4e04-91e4-5622e2856864 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:07.927410 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294daa3-2d83-417f-897b-8a9a99f3860e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:07.928295 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294daa3-2d83-417f-897b-8a9a99f3860e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:07.928728 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294daa3-2d83-417f-897b-8a9a99f3860e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:07.929152 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294daa3-2d83-417f-897b-8a9a99f3860e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:07.930269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f294daa3-2d83-417f-897b-8a9a99f3860e None admin] RBAC: Adding query filter params (role.id=9c013d602e9640b1883700191dcbd10a, user.id=424b8eb6a4f245bbb1a61de4b8ee9397, scope.system=all) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:07.930743 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f294daa3-2d83-417f-897b-8a9a99f3860e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:07.932450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f294daa3-2d83-417f-897b-8a9a99f3860e None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:07.938408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 153/308] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Sep 5 14:14:07 2022] GET /identity/v3/role_assignments?user.id=424b8eb6a4f245bbb1a61de4b8ee9397&scope.system=all&role.id=9c013d602e9640b1883700191dcbd10a => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:07.996212 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd24198-6030-47da-83bf-90b776cf88c7 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:08.036588 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68669134-2f41-49b5-baba-39b903189c3b None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:08.036875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68669134-2f41-49b5-baba-39b903189c3b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:08.037162 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68669134-2f41-49b5-baba-39b903189c3b None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:08.037382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:08.040058 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 154/309] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:14:08.045211 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d319f8-e6bd-4e08-955d-9c3c927db549 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:08.045507 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d319f8-e6bd-4e08-955d-9c3c927db549 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:08.045794 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d319f8-e6bd-4e08-955d-9c3c927db549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:08.047840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b8679-4462-448a-8fea-bf00a01fd73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:08.048514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b8679-4462-448a-8fea-bf00a01fd73d None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:08.048798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b8679-4462-448a-8fea-bf00a01fd73d None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:08.056544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b8679-4462-448a-8fea-bf00a01fd73d None admin] PATH_INFO: `/v3/users/08348ff876fa4490918498ac7e5822cb` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:08.056544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b8679-4462-448a-8fea-bf00a01fd73d None admin] RBAC: Authorizing `identity:get_user(user_id=08348ff876fa4490918498ac7e5822cb)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:08.056544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b8679-4462-448a-8fea-bf00a01fd73d None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:08.056544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 156/310] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:14:07 2022] GET /identity/v3/users/08348ff876fa4490918498ac7e5822cb => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:08.059858 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b8679-4462-448a-8fea-bf00a01fd73d None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:08.067686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-02d319f8-e6bd-4e08-955d-9c3c927db549 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:08.069282 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-02d319f8-e6bd-4e08-955d-9c3c927db549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:08.130299 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:08.151807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 155/311] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:08.184074 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b8679-4462-448a-8fea-bf00a01fd73d None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:08.219745 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3d899-98c9-48a5-899c-f7eb011b1532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:08.220808 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3d899-98c9-48a5-899c-f7eb011b1532 None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:08.221318 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3d899-98c9-48a5-899c-f7eb011b1532 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:08.221763 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3d899-98c9-48a5-899c-f7eb011b1532 None admin] PATH_INFO: `/v3/system/users/08348ff876fa4490918498ac7e5822cb/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:08.223360 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3d899-98c9-48a5-899c-f7eb011b1532 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=08348ff876fa4490918498ac7e5822cb, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:08.224959 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3d899-98c9-48a5-899c-f7eb011b1532 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:08.320104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 157/312] 10.222.0.15 () {58 vars in 1334 bytes} [Mon Sep 5 14:14:08 2022] PUT /identity/v3/system/users/08348ff876fa4490918498ac7e5822cb/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:14:08.421477 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d319f8-e6bd-4e08-955d-9c3c927db549 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:08.527092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d319f8-e6bd-4e08-955d-9c3c927db549 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:08.561668 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9285c7-868d-44fe-8add-6190337ad8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:08.562307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9285c7-868d-44fe-8add-6190337ad8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:08.562590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9285c7-868d-44fe-8add-6190337ad8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:08.562889 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9285c7-868d-44fe-8add-6190337ad8f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:08.574293 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9285c7-868d-44fe-8add-6190337ad8f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:08.579260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-dd9285c7-868d-44fe-8add-6190337ad8f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:14:08.579260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:08.579260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:08.579260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:08.579260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:08.579260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:08.579260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:08.579260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:08.579260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:08.579260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:08.579260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:08.579260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:08.579260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:08.579260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:08.579260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:08.579260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:08.580164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:08.580164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:08.580164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:08.580164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:08.580164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:08.580164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:08.580164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:08.580164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:08.580164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:08.580164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:08.580164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:08.580164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:08.580164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:08.580164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:08.580164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:08.580164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:08.580960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:08.580960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:08.580960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:08.580960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:08.580960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:08.580960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:08.580960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:08.580960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:08.580960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:08.580960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:08.580960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:14:08.580960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:08.580960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 156/313] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:08.586522 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3d899-98c9-48a5-899c-f7eb011b1532 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:08.646502 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:08.698861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3d899-98c9-48a5-899c-f7eb011b1532 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:08.736352 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d3380-001d-4508-91a8-02ef21bb31d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:08.736975 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d3380-001d-4508-91a8-02ef21bb31d6 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:08.737258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d3380-001d-4508-91a8-02ef21bb31d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:08.737535 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d3380-001d-4508-91a8-02ef21bb31d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:08.738201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d3380-001d-4508-91a8-02ef21bb31d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:08.738505 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d3380-001d-4508-91a8-02ef21bb31d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:08.739934 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d3380-001d-4508-91a8-02ef21bb31d6 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:08.748791 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 158/314] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:14:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:08.753957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9285c7-868d-44fe-8add-6190337ad8f3 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:08.798653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:08.859147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9285c7-868d-44fe-8add-6190337ad8f3 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:08.893246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7134d4f-145a-4f44-ae7b-5d5fb5aef455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:08.893883 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7134d4f-145a-4f44-ae7b-5d5fb5aef455 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:08.894165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7134d4f-145a-4f44-ae7b-5d5fb5aef455 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:08.894494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7134d4f-145a-4f44-ae7b-5d5fb5aef455 None admin] PATH_INFO: `/v3/users/7d54389fbedc4d75ba9c8fdae74db9db` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:08.895718 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7134d4f-145a-4f44-ae7b-5d5fb5aef455 None admin] RBAC: Authorizing `identity:get_user(user_id=7d54389fbedc4d75ba9c8fdae74db9db)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:08.897229 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7134d4f-145a-4f44-ae7b-5d5fb5aef455 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:08.899133 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 157/315] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:14:08 2022] GET /identity/v3/users/7d54389fbedc4d75ba9c8fdae74db9db => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:08.905369 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d3380-001d-4508-91a8-02ef21bb31d6 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:09.089698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d3380-001d-4508-91a8-02ef21bb31d6 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:09.109486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:09.123607 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a2df8-9e1a-49c0-8ca1-d0bf3f619aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:09.124223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a2df8-9e1a-49c0-8ca1-d0bf3f619aca None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:09.124505 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a2df8-9e1a-49c0-8ca1-d0bf3f619aca None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:09.124842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a2df8-9e1a-49c0-8ca1-d0bf3f619aca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:09.192661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a2df8-9e1a-49c0-8ca1-d0bf3f619aca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:09.194128 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a2df8-9e1a-49c0-8ca1-d0bf3f619aca None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:09.292053 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 159/316] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:09.297650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7134d4f-145a-4f44-ae7b-5d5fb5aef455 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:09.405142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7134d4f-145a-4f44-ae7b-5d5fb5aef455 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:09.414000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:09.440398 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c9e54-8a5c-4d5a-a916-22facaa97768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:09.441029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c9e54-8a5c-4d5a-a916-22facaa97768 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:09.441307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c9e54-8a5c-4d5a-a916-22facaa97768 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:09.441574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c9e54-8a5c-4d5a-a916-22facaa97768 None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:09.442697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c9e54-8a5c-4d5a-a916-22facaa97768 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:09.444267 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c9e54-8a5c-4d5a-a916-22facaa97768 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:09.446213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 158/317] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:09 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:09.452409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a2df8-9e1a-49c0-8ca1-d0bf3f619aca None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:09.478097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:09.559059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a2df8-9e1a-49c0-8ca1-d0bf3f619aca None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:09.595500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ba49f-c47b-4c2f-9d23-adfdf44323a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:09.596169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ba49f-c47b-4c2f-9d23-adfdf44323a8 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:09.596410 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ba49f-c47b-4c2f-9d23-adfdf44323a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:09.596730 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ba49f-c47b-4c2f-9d23-adfdf44323a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:09.597548 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ba49f-c47b-4c2f-9d23-adfdf44323a8 None admin] RBAC: Adding query filter params (role.id=9c013d602e9640b1883700191dcbd10a, scope.project.id=e5e3e090ce8348c19e0ddc4994094302, user.id=7d54389fbedc4d75ba9c8fdae74db9db) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:09.597859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ba49f-c47b-4c2f-9d23-adfdf44323a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:09.599420 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ba49f-c47b-4c2f-9d23-adfdf44323a8 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:09.606701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 160/318] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:14:09 2022] GET /identity/v3/role_assignments?user.id=7d54389fbedc4d75ba9c8fdae74db9db&scope.project.id=e5e3e090ce8348c19e0ddc4994094302&role.id=9c013d602e9640b1883700191dcbd10a => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:09.607742 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d83dd-16d3-4cb8-b752-1472b4e500b7 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:09.608132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d83dd-16d3-4cb8-b752-1472b4e500b7 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:09.608488 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d83dd-16d3-4cb8-b752-1472b4e500b7 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:09.609995 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 159/319] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:09.617982 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eeb0ff-6e28-46de-8ddc-9310236b40a8 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:09.618294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eeb0ff-6e28-46de-8ddc-9310236b40a8 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:09.618556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eeb0ff-6e28-46de-8ddc-9310236b40a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:09.638192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-26eeb0ff-6e28-46de-8ddc-9310236b40a8 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:09.639472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-26eeb0ff-6e28-46de-8ddc-9310236b40a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:09.716535 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 161/320] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:09.894188 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d719034-b5fa-4a58-8a7b-b7f34f0e60c9 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:09.894693 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d719034-b5fa-4a58-8a7b-b7f34f0e60c9 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:09.894768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d719034-b5fa-4a58-8a7b-b7f34f0e60c9 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:09.896173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 160/321] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:09.902001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ae45d-841e-4263-86e2-37db72225729 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:09.902291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ae45d-841e-4263-86e2-37db72225729 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:09.902572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ae45d-841e-4263-86e2-37db72225729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:09.920794 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-b31ae45d-841e-4263-86e2-37db72225729 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:09.924893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b31ae45d-841e-4263-86e2-37db72225729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:09.931855 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:09.990128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d719034-b5fa-4a58-8a7b-b7f34f0e60c9 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:09.998893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 162/322] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:10.101876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d719034-b5fa-4a58-8a7b-b7f34f0e60c9 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:10.135953 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c32a2d-65f5-40c0-bddc-0506cd8a1674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:10.136585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c32a2d-65f5-40c0-bddc-0506cd8a1674 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:10.136870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c32a2d-65f5-40c0-bddc-0506cd8a1674 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:10.137148 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c32a2d-65f5-40c0-bddc-0506cd8a1674 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:10.138241 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c32a2d-65f5-40c0-bddc-0506cd8a1674 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:10.139729 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c32a2d-65f5-40c0-bddc-0506cd8a1674 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:10.141468 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 161/323] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:10.146813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ae45d-841e-4263-86e2-37db72225729 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:10.245895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ae45d-841e-4263-86e2-37db72225729 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:10.268632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c32a2d-65f5-40c0-bddc-0506cd8a1674 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:10.283592 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b8988-7fef-4ff8-97c4-a632a471ebdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:10.284708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b8988-7fef-4ff8-97c4-a632a471ebdd None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:10.285512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b8988-7fef-4ff8-97c4-a632a471ebdd None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:10.286262 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b8988-7fef-4ff8-97c4-a632a471ebdd None admin] PATH_INFO: `/v3/groups` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:10.287117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b8988-7fef-4ff8-97c4-a632a471ebdd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:10.289195 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b8988-7fef-4ff8-97c4-a632a471ebdd None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:10.316064 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:10.369499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c32a2d-65f5-40c0-bddc-0506cd8a1674 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:10.402831 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 163/324] 10.222.0.15 () {60 vars in 1134 bytes} [Mon Sep 5 14:14:10 2022] POST /identity/v3/groups => generated 226 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:14:10.404840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb751243-b485-437f-ae64-f80ffb9190df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:10.405608 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb751243-b485-437f-ae64-f80ffb9190df None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:10.405892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb751243-b485-437f-ae64-f80ffb9190df None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:10.406166 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb751243-b485-437f-ae64-f80ffb9190df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:10.419834 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb751243-b485-437f-ae64-f80ffb9190df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:10.426033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-eb751243-b485-437f-ae64-f80ffb9190df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:14:10.426033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:10.426033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:10.426033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:10.426033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:10.426033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:10.426033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:10.426033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:10.426033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:10.426033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:10.426033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:10.426033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:10.426033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:10.426033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:10.426033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:10.426033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:10.427096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:10.427096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:10.427096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:10.427096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:10.427096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:10.427096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:10.427096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:10.427096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:10.427096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:10.427096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:10.427096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:10.427096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:10.427096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:10.427096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:10.427096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:10.427096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:10.427903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:10.427903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:10.427903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:10.427903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:10.427903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:10.427903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:10.427903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:10.427903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:10.427903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:10.427903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:10.427903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:14:10.427903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:10.427903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 162/325] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:10.435525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b8988-7fef-4ff8-97c4-a632a471ebdd None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:10.440847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:10.461255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f8781-f62a-4c81-a67d-3268d8912f36 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:10.461602 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f8781-f62a-4c81-a67d-3268d8912f36 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:10.461814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f8781-f62a-4c81-a67d-3268d8912f36 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:10.464345 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 163/326] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:14:10.470971 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ec97f-68c0-488c-9e21-eeb7c9cc57ca None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:10.471261 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ec97f-68c0-488c-9e21-eeb7c9cc57ca None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:10.471810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ec97f-68c0-488c-9e21-eeb7c9cc57ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:10.493020 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-e21ec97f-68c0-488c-9e21-eeb7c9cc57ca None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:10.494834 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e21ec97f-68c0-488c-9e21-eeb7c9cc57ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:10.554962 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b8988-7fef-4ff8-97c4-a632a471ebdd None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:10.572741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 164/327] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:14:10.593932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f41a7-5186-4851-b050-2492c95dbf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:10.594556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f41a7-5186-4851-b050-2492c95dbf18 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:10.594873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f41a7-5186-4851-b050-2492c95dbf18 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:10.595152 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f41a7-5186-4851-b050-2492c95dbf18 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:10.595816 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f41a7-5186-4851-b050-2492c95dbf18 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:10.596321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f41a7-5186-4851-b050-2492c95dbf18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:10.597487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f41a7-5186-4851-b050-2492c95dbf18 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:10.605982 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 164/328] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:14:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:10.611627 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ec97f-68c0-488c-9e21-eeb7c9cc57ca None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:10.648023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:10.726508 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ec97f-68c0-488c-9e21-eeb7c9cc57ca None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:10.763600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a446c-293c-4e4a-b62f-f3b9ae53365c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:10.764235 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a446c-293c-4e4a-b62f-f3b9ae53365c None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:10.764511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a446c-293c-4e4a-b62f-f3b9ae53365c None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:10.764779 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a446c-293c-4e4a-b62f-f3b9ae53365c None admin] PATH_INFO: `/v3/users/424b8eb6a4f245bbb1a61de4b8ee9397` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:10.765902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a446c-293c-4e4a-b62f-f3b9ae53365c None admin] RBAC: Authorizing `identity:get_user(user_id=424b8eb6a4f245bbb1a61de4b8ee9397)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:10.767453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a446c-293c-4e4a-b62f-f3b9ae53365c None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:10.769358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 165/329] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:14:10 2022] GET /identity/v3/users/424b8eb6a4f245bbb1a61de4b8ee9397 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:10.775707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f41a7-5186-4851-b050-2492c95dbf18 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:10.844858 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a446c-293c-4e4a-b62f-f3b9ae53365c None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:10.850082 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:10.895385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f41a7-5186-4851-b050-2492c95dbf18 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:10.945187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953b31d-42bb-48a4-8731-d4cb385e118c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:10.946228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953b31d-42bb-48a4-8731-d4cb385e118c None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:10.946949 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953b31d-42bb-48a4-8731-d4cb385e118c None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:10.947433 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953b31d-42bb-48a4-8731-d4cb385e118c None admin] PATH_INFO: `/v3/system/users/424b8eb6a4f245bbb1a61de4b8ee9397/roles/9c013d602e9640b1883700191dcbd10a` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:10.949098 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953b31d-42bb-48a4-8731-d4cb385e118c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=424b8eb6a4f245bbb1a61de4b8ee9397, role_id=9c013d602e9640b1883700191dcbd10a)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:10.950828 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953b31d-42bb-48a4-8731-d4cb385e118c None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:10.977698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a446c-293c-4e4a-b62f-f3b9ae53365c None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:10.979963 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:11.020017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f0c0b-b931-4546-bad0-ec82f6bd2aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:11.020717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f0c0b-b931-4546-bad0-ec82f6bd2aeb None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:11.020996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f0c0b-b931-4546-bad0-ec82f6bd2aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:11.021332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f0c0b-b931-4546-bad0-ec82f6bd2aeb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:11.032302 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f0c0b-b931-4546-bad0-ec82f6bd2aeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:11.037388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-543f0c0b-b931-4546-bad0-ec82f6bd2aeb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:11.037388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:11.037388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:11.037388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:11.037388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:11.037388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:11.037388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:11.037388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:11.037388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:11.037388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:11.037388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:11.037388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:11.037388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:11.037388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:11.037388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:11.037388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:11.038362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:11.038362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:11.038362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:11.038362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:11.038362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:11.038362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:11.038362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:11.038362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:11.038362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:11.038362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:11.038362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:11.038362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:11.038362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:11.038362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:11.038362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:11.038362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:11.039216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:11.039216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:11.039216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:11.039216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:11.039216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:11.039216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:11.039216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:11.039216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:11.039216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:11.039216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:11.039216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:11.039216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:11.039216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 166/330] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:11.046678 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f0c0b-b931-4546-bad0-ec82f6bd2aeb None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:11.051315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 165/331] 10.222.0.15 () {58 vars in 1334 bytes} [Mon Sep 5 14:14:10 2022] PUT /identity/v3/system/users/424b8eb6a4f245bbb1a61de4b8ee9397/roles/9c013d602e9640b1883700191dcbd10a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:14:11.176793 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f0c0b-b931-4546-bad0-ec82f6bd2aeb None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:11.218752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d6c7e-41c8-4897-8d3a-ada25426b5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:11.219441 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d6c7e-41c8-4897-8d3a-ada25426b5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:11.219720 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d6c7e-41c8-4897-8d3a-ada25426b5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:11.219992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d6c7e-41c8-4897-8d3a-ada25426b5e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:11.220708 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d6c7e-41c8-4897-8d3a-ada25426b5e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:11.221016 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d6c7e-41c8-4897-8d3a-ada25426b5e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:11.222490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d6c7e-41c8-4897-8d3a-ada25426b5e8 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:11.233848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 167/332] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:14:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:11.239553 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953b31d-42bb-48a4-8731-d4cb385e118c None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:11.362671 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953b31d-42bb-48a4-8731-d4cb385e118c None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:11.399936 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:11.406949 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8fe06-a89d-48f9-94c3-5b4b182c6e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:11.407769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8fe06-a89d-48f9-94c3-5b4b182c6e2a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:11.408200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8fe06-a89d-48f9-94c3-5b4b182c6e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:11.414754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8fe06-a89d-48f9-94c3-5b4b182c6e2a None admin] PATH_INFO: `/v3/users/08348ff876fa4490918498ac7e5822cb` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:11.414754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8fe06-a89d-48f9-94c3-5b4b182c6e2a None admin] RBAC: Authorizing `identity:get_user(user_id=08348ff876fa4490918498ac7e5822cb)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:11.414754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8fe06-a89d-48f9-94c3-5b4b182c6e2a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:11.414754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 166/333] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:14:11 2022] GET /identity/v3/users/08348ff876fa4490918498ac7e5822cb => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:11.419576 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d6c7e-41c8-4897-8d3a-ada25426b5e8 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:11.460327 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:11.545580 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d6c7e-41c8-4897-8d3a-ada25426b5e8 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:11.588891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03113308-40e7-426f-a5da-2b1fccabab08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:11.589507 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03113308-40e7-426f-a5da-2b1fccabab08 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:11.589785 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03113308-40e7-426f-a5da-2b1fccabab08 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:11.590054 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03113308-40e7-426f-a5da-2b1fccabab08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:11.590776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03113308-40e7-426f-a5da-2b1fccabab08 None admin] RBAC: Adding query filter params (role.id=5812e6538f3f4bdfbfbe11cd25c02957, user.id=08348ff876fa4490918498ac7e5822cb, scope.system=all) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:11.591098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03113308-40e7-426f-a5da-2b1fccabab08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:11.592538 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03113308-40e7-426f-a5da-2b1fccabab08 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:11.610046 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 168/334] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Sep 5 14:14:11 2022] GET /identity/v3/role_assignments?user.id=08348ff876fa4490918498ac7e5822cb&scope.system=all&role.id=5812e6538f3f4bdfbfbe11cd25c02957 => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:13.224478 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77645b7e-90f6-4f10-a4a8-08b48f405ccf None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:13.225484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77645b7e-90f6-4f10-a4a8-08b48f405ccf None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:13.225973 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77645b7e-90f6-4f10-a4a8-08b48f405ccf None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:13.227561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 167/335] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:13.234701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e79f3-1c0a-4013-8b27-1e9fb674ba82 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:13.235600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e79f3-1c0a-4013-8b27-1e9fb674ba82 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:13.236079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e79f3-1c0a-4013-8b27-1e9fb674ba82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:13.245580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da71793-b3b5-44ad-932a-85b0cd417929 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:13.245580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da71793-b3b5-44ad-932a-85b0cd417929 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:13.245580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da71793-b3b5-44ad-932a-85b0cd417929 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:13.245580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 168/336] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:13.258821 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63645116-64e8-4f9d-8015-295031417b55 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:13.258821 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63645116-64e8-4f9d-8015-295031417b55 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:13.258821 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63645116-64e8-4f9d-8015-295031417b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:13.264080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-8a9e79f3-1c0a-4013-8b27-1e9fb674ba82 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:13.265533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8a9e79f3-1c0a-4013-8b27-1e9fb674ba82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:13.295669 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-63645116-64e8-4f9d-8015-295031417b55 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:13.296906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-63645116-64e8-4f9d-8015-295031417b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:13.379870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 169/337] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:13.403938 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 169/338] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 05 14:14:13.417420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de79cab-6b05-45da-9901-992c319668b6 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:13.417991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de79cab-6b05-45da-9901-992c319668b6 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:13.418466 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de79cab-6b05-45da-9901-992c319668b6 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:13.420026 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 170/339] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:13.425913 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60495c60-8eeb-4f22-ade3-be1e52f4224c None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:13.426199 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60495c60-8eeb-4f22-ade3-be1e52f4224c None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:13.426476 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60495c60-8eeb-4f22-ade3-be1e52f4224c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:13.455790 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-60495c60-8eeb-4f22-ade3-be1e52f4224c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:13.456996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-60495c60-8eeb-4f22-ade3-be1e52f4224c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:13.537063 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 170/340] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:13.820643 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de79cab-6b05-45da-9901-992c319668b6 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:13.836661 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:13.840554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60495c60-8eeb-4f22-ade3-be1e52f4224c None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:13.964271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:14.041684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60495c60-8eeb-4f22-ade3-be1e52f4224c None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:14.042737 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de79cab-6b05-45da-9901-992c319668b6 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:14.084521 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125c9de-fa39-4586-b670-958f94c418e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:14.088866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125c9de-fa39-4586-b670-958f94c418e4 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:14.089064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125c9de-fa39-4586-b670-958f94c418e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:14.089345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125c9de-fa39-4586-b670-958f94c418e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:14.105274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125c9de-fa39-4586-b670-958f94c418e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:14.110494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-2125c9de-fa39-4586-b670-958f94c418e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:14:14.110494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:14.110494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:14.110494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:14.110494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:14.110494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:14.110494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:14.110494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:14.110494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:14.110494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:14.110494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:14.110494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:14.110494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:14.110494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:14.110494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:14.110494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:14.111417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:14.111417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:14.111417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:14.111417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:14.111417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:14.111417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:14.111417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:14.111417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:14.111417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:14.111417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:14.111417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:14.111417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:14.111417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:14.111417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:14.111417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:14.111417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:14.113107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:14.113107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:14.113107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:14.113107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:14.113107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:14.113107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:14.113107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:14.113107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:14.113107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:14.113107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:14.113107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:14:14.113107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:14.113107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 171/341] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:14:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:14.113107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125c9de-fa39-4586-b670-958f94c418e4 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:14.158626 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc082bb-5a9f-4613-afe1-f132646a7482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:14.159749 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc082bb-5a9f-4613-afe1-f132646a7482 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:14.161828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc082bb-5a9f-4613-afe1-f132646a7482 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:14.162364 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc082bb-5a9f-4613-afe1-f132646a7482 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:14.175482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc082bb-5a9f-4613-afe1-f132646a7482 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:14.181144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-3fc082bb-5a9f-4613-afe1-f132646a7482 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:14:14.181144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:14.181144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:14.181144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:14.181144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:14.181144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:14.181144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:14.181144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:14.181144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:14.181144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:14.181144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:14.181144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:14.181144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:14.181144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:14.181144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:14.181144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:14.182551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:14.182551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:14.182551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:14.182551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:14.182551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:14.182551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:14.182551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:14.182551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:14.182551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:14.182551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:14.182551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:14.182551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:14.182551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:14.182551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:14.182551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:14.182551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:14.183475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:14.183475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:14.183475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:14.183475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:14.183475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:14.183475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:14.183475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:14.183475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:14.183475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:14.183475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:14.183475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 14:14:14.183475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:14.186272 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 171/342] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:14.187837 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f44ae-7155-45f1-972c-11c488f86f72 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:14.189879 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f44ae-7155-45f1-972c-11c488f86f72 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:14.191071 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f44ae-7155-45f1-972c-11c488f86f72 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:14.192794 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 172/343] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:14.193843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f44ae-7155-45f1-972c-11c488f86f72 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:14.244471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125c9de-fa39-4586-b670-958f94c418e4 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:14.263280 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:14.283283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675810f8-54b0-46f6-a55f-e287ab1107b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:14.283973 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675810f8-54b0-46f6-a55f-e287ab1107b1 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:14.284206 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:14.284623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675810f8-54b0-46f6-a55f-e287ab1107b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:14.284623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675810f8-54b0-46f6-a55f-e287ab1107b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:14.295623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675810f8-54b0-46f6-a55f-e287ab1107b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:14.300781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-675810f8-54b0-46f6-a55f-e287ab1107b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:14.300781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:14.300781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:14.300781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:14.300781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:14.300781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:14.300781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:14.300781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:14.300781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:14.300781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:14.300781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:14.300781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:14.300781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:14.300781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:14.300781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:14.300781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:14.301729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:14.301729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:14.301729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:14.301729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:14.301729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:14.301729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:14.301729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:14.301729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:14.301729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:14.301729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:14.301729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:14.301729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:14.301729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:14.301729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:14.301729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:14.301729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:14.302741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:14.302741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:14.302741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:14.302741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:14.302741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:14.302741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:14.302741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:14.302741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:14.302741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:14.302741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:14.302741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:14.302741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:14.302741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 172/344] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:14.304428 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675810f8-54b0-46f6-a55f-e287ab1107b1 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:14.323838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f44ae-7155-45f1-972c-11c488f86f72 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:14.362101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd7fec-4604-402e-8df9-40b7b6ddf69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:14.363021 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd7fec-4604-402e-8df9-40b7b6ddf69a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:14.363491 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd7fec-4604-402e-8df9-40b7b6ddf69a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:14.363964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd7fec-4604-402e-8df9-40b7b6ddf69a None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:14.364904 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd7fec-4604-402e-8df9-40b7b6ddf69a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:14.365390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd7fec-4604-402e-8df9-40b7b6ddf69a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:14.367077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd7fec-4604-402e-8df9-40b7b6ddf69a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:14.381539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 173/345] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:14.383118 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354ff73-89db-4223-becc-cd47440b6464 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:14.383589 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354ff73-89db-4223-becc-cd47440b6464 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:14.384040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354ff73-89db-4223-becc-cd47440b6464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:14.403827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-9354ff73-89db-4223-becc-cd47440b6464 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:14.405502 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9354ff73-89db-4223-becc-cd47440b6464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:14.414129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675810f8-54b0-46f6-a55f-e287ab1107b1 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:14.457316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e2f58-f69d-4246-a590-a02d23247b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:14.458032 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e2f58-f69d-4246-a590-a02d23247b8c None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:14.458412 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e2f58-f69d-4246-a590-a02d23247b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:14.459722 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e2f58-f69d-4246-a590-a02d23247b8c None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:14.459722 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e2f58-f69d-4246-a590-a02d23247b8c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:14.459910 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e2f58-f69d-4246-a590-a02d23247b8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:14.461273 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e2f58-f69d-4246-a590-a02d23247b8c None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:14.471903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 173/346] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:14:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:14.472516 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e2f58-f69d-4246-a590-a02d23247b8c None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:14.498575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 174/347] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:14.499653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354ff73-89db-4223-becc-cd47440b6464 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:14.527137 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:14.575031 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:14.604202 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e2f58-f69d-4246-a590-a02d23247b8c None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:14.624097 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354ff73-89db-4223-becc-cd47440b6464 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:14.645886 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803552e-5efe-4e03-b1bf-9903df4f6af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:14.646503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803552e-5efe-4e03-b1bf-9903df4f6af9 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:14.646809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803552e-5efe-4e03-b1bf-9903df4f6af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:14.647099 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803552e-5efe-4e03-b1bf-9903df4f6af9 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:14.647835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803552e-5efe-4e03-b1bf-9903df4f6af9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:14.648142 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803552e-5efe-4e03-b1bf-9903df4f6af9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:14.649517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803552e-5efe-4e03-b1bf-9903df4f6af9 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:14.659369 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 174/348] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:14:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:14.660775 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74abf62-6c7b-4a42-a57e-03e1f57012d3 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:14.661292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74abf62-6c7b-4a42-a57e-03e1f57012d3 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:14.661419 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74abf62-6c7b-4a42-a57e-03e1f57012d3 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:14.662630 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 175/349] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:14.667731 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74abf62-6c7b-4a42-a57e-03e1f57012d3 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:14.671124 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed683185-436e-47d1-84a0-ca5c1b742f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:14.672004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed683185-436e-47d1-84a0-ca5c1b742f5a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:14.673393 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed683185-436e-47d1-84a0-ca5c1b742f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:14.673846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed683185-436e-47d1-84a0-ca5c1b742f5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:14.776337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed683185-436e-47d1-84a0-ca5c1b742f5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:14.778879 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed683185-436e-47d1-84a0-ca5c1b742f5a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:14.795659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74abf62-6c7b-4a42-a57e-03e1f57012d3 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:14.797433 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:14.842876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a0eb1-cbf1-4240-a9de-08d006be46ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:14.844223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a0eb1-cbf1-4240-a9de-08d006be46ce None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:14.844944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a0eb1-cbf1-4240-a9de-08d006be46ce None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:14.845217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a0eb1-cbf1-4240-a9de-08d006be46ce None admin] PATH_INFO: `/v3/users/424b8eb6a4f245bbb1a61de4b8ee9397` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:14.846434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a0eb1-cbf1-4240-a9de-08d006be46ce None admin] RBAC: Authorizing `identity:get_user(user_id=424b8eb6a4f245bbb1a61de4b8ee9397)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:14.848091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a0eb1-cbf1-4240-a9de-08d006be46ce None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:14.850029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 176/350] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:14:14 2022] GET /identity/v3/users/424b8eb6a4f245bbb1a61de4b8ee9397 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:14.850845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a0eb1-cbf1-4240-a9de-08d006be46ce None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:14.863973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:14.909922 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 175/351] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:14.911308 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb7e4a-99bd-4d34-aaae-f05946fbf115 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:14.911776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb7e4a-99bd-4d34-aaae-f05946fbf115 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:14.912168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb7e4a-99bd-4d34-aaae-f05946fbf115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:14.936454 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-a6cb7e4a-99bd-4d34-aaae-f05946fbf115 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:14.937650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a6cb7e4a-99bd-4d34-aaae-f05946fbf115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:14.981087 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a0eb1-cbf1-4240-a9de-08d006be46ce None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:15.018399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 176/352] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:15.019401 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb7e4a-99bd-4d34-aaae-f05946fbf115 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:15.025512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586064e4-091f-46fe-9af5-fe2e79df048c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:15.026170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586064e4-091f-46fe-9af5-fe2e79df048c None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:15.026448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586064e4-091f-46fe-9af5-fe2e79df048c None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:15.026768 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586064e4-091f-46fe-9af5-fe2e79df048c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:15.084192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:15.124043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586064e4-091f-46fe-9af5-fe2e79df048c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:15.126736 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586064e4-091f-46fe-9af5-fe2e79df048c None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:15.135455 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:15.142430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb7e4a-99bd-4d34-aaae-f05946fbf115 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:15.192533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c5a2b8-3520-4b6c-b686-d2ef479cb58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:15.193094 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5a2b8-3520-4b6c-b686-d2ef479cb58c None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:15.193379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5a2b8-3520-4b6c-b686-d2ef479cb58c None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:15.193656 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5a2b8-3520-4b6c-b686-d2ef479cb58c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:15.211416 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c5a2b8-3520-4b6c-b686-d2ef479cb58c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:15.218049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-b3c5a2b8-3520-4b6c-b686-d2ef479cb58c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:15.218049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:15.218049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:15.218049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:15.218049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:15.218049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:15.218049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:15.218049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:15.218049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:15.218049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:15.218049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:15.218049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:15.218049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:15.218049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:15.218049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:15.218049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:15.219040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:15.219040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:15.219040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:15.219040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:15.219040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:15.219040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:15.219040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:15.219040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:15.219040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:15.219040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:15.219040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:15.219040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:15.219040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:15.219040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:15.219040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:15.219040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:15.219845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:15.219845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:15.219845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:15.219845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:15.219845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:15.219845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:15.219845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:15.219845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:15.219845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:15.219845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:15.219845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:15.219845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:15.219845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 177/353] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:15.220501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c5a2b8-3520-4b6c-b686-d2ef479cb58c None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:15.269206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 177/354] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:15.270073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586064e4-091f-46fe-9af5-fe2e79df048c None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:15.355190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c5a2b8-3520-4b6c-b686-d2ef479cb58c None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:15.359046 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:15.394200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586064e4-091f-46fe-9af5-fe2e79df048c None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:15.396703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84193c82-dabe-449f-966a-6576c0503a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:15.397319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84193c82-dabe-449f-966a-6576c0503a48 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:15.397604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84193c82-dabe-449f-966a-6576c0503a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:15.397874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84193c82-dabe-449f-966a-6576c0503a48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:15.398611 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84193c82-dabe-449f-966a-6576c0503a48 None admin] RBAC: Adding query filter params (role.id=9c013d602e9640b1883700191dcbd10a, user.id=424b8eb6a4f245bbb1a61de4b8ee9397, scope.system=all) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:15.398952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84193c82-dabe-449f-966a-6576c0503a48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:15.400988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84193c82-dabe-449f-966a-6576c0503a48 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:15.417800 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 178/355] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Sep 5 14:14:15 2022] GET /identity/v3/role_assignments?user.id=424b8eb6a4f245bbb1a61de4b8ee9397&scope.system=all&role.id=9c013d602e9640b1883700191dcbd10a => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:15.419423 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce0b37-96e7-4880-ac8a-5435e4b3801b None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:15.419423 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce0b37-96e7-4880-ac8a-5435e4b3801b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:15.419722 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce0b37-96e7-4880-ac8a-5435e4b3801b None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:15.421396 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 179/356] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:15.422122 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce0b37-96e7-4880-ac8a-5435e4b3801b None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:15.438816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:15.441534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d6cc7-3ef5-4b16-8664-2db7ff20a30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:15.442187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d6cc7-3ef5-4b16-8664-2db7ff20a30a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:15.443009 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d6cc7-3ef5-4b16-8664-2db7ff20a30a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:15.443009 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d6cc7-3ef5-4b16-8664-2db7ff20a30a None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:15.445866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d6cc7-3ef5-4b16-8664-2db7ff20a30a None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:15.448261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d6cc7-3ef5-4b16-8664-2db7ff20a30a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:15.451411 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 178/357] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:15 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:15.451411 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d6cc7-3ef5-4b16-8664-2db7ff20a30a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:15.564418 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce0b37-96e7-4880-ac8a-5435e4b3801b None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:15.607052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d6cc7-3ef5-4b16-8664-2db7ff20a30a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:15.610179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33eda9-3d8c-428d-b153-1c1d4f86dd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:15.610888 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33eda9-3d8c-428d-b153-1c1d4f86dd37 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:15.611174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33eda9-3d8c-428d-b153-1c1d4f86dd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:15.611447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33eda9-3d8c-428d-b153-1c1d4f86dd37 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:15.612119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33eda9-3d8c-428d-b153-1c1d4f86dd37 None admin] RBAC: Adding query filter params (name=member) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:15.612427 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33eda9-3d8c-428d-b153-1c1d4f86dd37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:15.613824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33eda9-3d8c-428d-b153-1c1d4f86dd37 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:15.624586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 180/358] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:14:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:15.625977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33eda9-3d8c-428d-b153-1c1d4f86dd37 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:15.649946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecb865-4f12-414b-b1bf-a781735daa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:15.651174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecb865-4f12-414b-b1bf-a781735daa75 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:15.651425 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecb865-4f12-414b-b1bf-a781735daa75 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:15.651700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecb865-4f12-414b-b1bf-a781735daa75 None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:15.652950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cecb865-4f12-414b-b1bf-a781735daa75 None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:15.654394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cecb865-4f12-414b-b1bf-a781735daa75 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:15.656299 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 179/359] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:15 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:15.657472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fc722-9ea7-4941-bb7e-7b5ef696599d None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:15.657834 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fc722-9ea7-4941-bb7e-7b5ef696599d None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:15.658148 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fc722-9ea7-4941-bb7e-7b5ef696599d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:15.671751 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:15.677690 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-031fc722-9ea7-4941-bb7e-7b5ef696599d None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:15.679142 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-031fc722-9ea7-4941-bb7e-7b5ef696599d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:15.727156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:15.757673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33eda9-3d8c-428d-b153-1c1d4f86dd37 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:15.762868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 180/360] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:15.763558 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fc722-9ea7-4941-bb7e-7b5ef696599d None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:15.807774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b4b81-55ab-4aa7-8c96-fca166f849e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:15.808859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b4b81-55ab-4aa7-8c96-fca166f849e9 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:15.809378 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b4b81-55ab-4aa7-8c96-fca166f849e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:15.809854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b4b81-55ab-4aa7-8c96-fca166f849e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:15.826019 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b4b81-55ab-4aa7-8c96-fca166f849e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:15.834103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-4c4b4b81-55ab-4aa7-8c96-fca166f849e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:14:15.834103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:15.834103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:15.834103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:15.834103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:15.834103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:15.834103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:15.834103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:15.834103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:15.834103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:15.834103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:15.834103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:15.834103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:15.834103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:15.834103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:15.834103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:15.835113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:15.835113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:15.835113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:15.835113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:15.835113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:15.835113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:15.835113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:15.835113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:15.835113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:15.835113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:15.835113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:15.835113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:15.835113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:15.835113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:15.835113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:15.835113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:15.835930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:15.835930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:15.835930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:15.835930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:15.835930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:15.835930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:15.835930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:15.835930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:15.835930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:15.835930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:15.835930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:14:15.835930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:15.837042 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 181/361] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:14:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:15.837383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b4b81-55ab-4aa7-8c96-fca166f849e9 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:15.884723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fc722-9ea7-4941-bb7e-7b5ef696599d None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:15.932480 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0b759-f003-4a41-948a-15794c8da7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:15.933133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0b759-f003-4a41-948a-15794c8da7e5 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:15.933423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0b759-f003-4a41-948a-15794c8da7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:15.933698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0b759-f003-4a41-948a-15794c8da7e5 None admin] PATH_INFO: `/v3/groups/bb53a81c848a4c60968050d0c87a771d` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:15.937767 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:15.940699 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0b759-f003-4a41-948a-15794c8da7e5 None admin] RBAC: Authorizing `identity:get_group(group_id=bb53a81c848a4c60968050d0c87a771d)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:15.942296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0b759-f003-4a41-948a-15794c8da7e5 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:15.945823 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 181/362] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 5 14:14:15 2022] GET /identity/v3/groups/bb53a81c848a4c60968050d0c87a771d => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:15.945823 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0b759-f003-4a41-948a-15794c8da7e5 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:15.979513 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b4b81-55ab-4aa7-8c96-fca166f849e9 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:16.023161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa504907-92d3-4c34-a253-a77f4a6f66dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:16.023787 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa504907-92d3-4c34-a253-a77f4a6f66dc None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:16.024091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa504907-92d3-4c34-a253-a77f4a6f66dc None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:16.024371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa504907-92d3-4c34-a253-a77f4a6f66dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:16.027324 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:16.077178 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0b759-f003-4a41-948a-15794c8da7e5 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:16.113155 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa504907-92d3-4c34-a253-a77f4a6f66dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:16.114688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa504907-92d3-4c34-a253-a77f4a6f66dc None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:16.119434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3805f6-7433-428b-a975-1c25eeab48c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:16.120294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3805f6-7433-428b-a975-1c25eeab48c8 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:16.120577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3805f6-7433-428b-a975-1c25eeab48c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:16.120853 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3805f6-7433-428b-a975-1c25eeab48c8 None admin] PATH_INFO: `/v3/groups/bb53a81c848a4c60968050d0c87a771d` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:16.122648 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3805f6-7433-428b-a975-1c25eeab48c8 None admin] RBAC: Authorizing `identity:get_group(group_id=bb53a81c848a4c60968050d0c87a771d)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:16.124167 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3805f6-7433-428b-a975-1c25eeab48c8 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:16.125766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 182/363] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 5 14:14:15 2022] GET /identity/v3/groups/bb53a81c848a4c60968050d0c87a771d => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:16.126626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3805f6-7433-428b-a975-1c25eeab48c8 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:16.204977 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:16.235717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 182/364] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:16.235717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa504907-92d3-4c34-a253-a77f4a6f66dc None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:16.243271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3805f6-7433-428b-a975-1c25eeab48c8 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:16.273770 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:16.288526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf473a3-1afa-47c0-8caf-1641b2b03f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:16.289195 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf473a3-1afa-47c0-8caf-1641b2b03f05 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:16.293119 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf473a3-1afa-47c0-8caf-1641b2b03f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:16.293119 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf473a3-1afa-47c0-8caf-1641b2b03f05 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:16.293119 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf473a3-1afa-47c0-8caf-1641b2b03f05 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:16.293119 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf473a3-1afa-47c0-8caf-1641b2b03f05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:16.293119 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf473a3-1afa-47c0-8caf-1641b2b03f05 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:16.300796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 183/365] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:16.301596 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf473a3-1afa-47c0-8caf-1641b2b03f05 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:16.358354 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa504907-92d3-4c34-a253-a77f4a6f66dc None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:16.402620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f504cd-509b-4e8a-a589-a5346e72d721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:16.403304 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f504cd-509b-4e8a-a589-a5346e72d721 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:16.403583 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f504cd-509b-4e8a-a589-a5346e72d721 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:16.404615 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f504cd-509b-4e8a-a589-a5346e72d721 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:16.404615 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f504cd-509b-4e8a-a589-a5346e72d721 None admin] RBAC: Adding query filter params (group.id=bb53a81c848a4c60968050d0c87a771d, role.id=1182ffc3e9f9477db2003500b7bfd402, scope.project.id=2269c4a7f991405497de468e3a9a745a) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:16.404924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f504cd-509b-4e8a-a589-a5346e72d721 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:16.406333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f504cd-509b-4e8a-a589-a5346e72d721 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:16.413340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 183/366] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 5 14:14:16 2022] GET /identity/v3/role_assignments?group.id=bb53a81c848a4c60968050d0c87a771d&scope.project.id=2269c4a7f991405497de468e3a9a745a&role.id=1182ffc3e9f9477db2003500b7bfd402 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:16.413340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f504cd-509b-4e8a-a589-a5346e72d721 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:16.418397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf473a3-1afa-47c0-8caf-1641b2b03f05 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:16.455831 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:16.464776 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8338df6-ee74-4d27-add1-be4480414a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:16.466187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8338df6-ee74-4d27-add1-be4480414a2b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:16.466187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8338df6-ee74-4d27-add1-be4480414a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:16.466187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8338df6-ee74-4d27-add1-be4480414a2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:16.481339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8338df6-ee74-4d27-add1-be4480414a2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:16.487496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-d8338df6-ee74-4d27-add1-be4480414a2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:14:16.487496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:16.487496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:16.487496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:16.487496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:16.487496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:16.487496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:16.487496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:16.487496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:16.487496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:16.487496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:16.487496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:16.487496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:16.487496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:16.487496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:16.487496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:16.488606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:16.488606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:16.488606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:16.488606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:16.488606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:16.488606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:16.488606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:16.488606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:16.488606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:16.488606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:16.488606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:16.488606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:16.488606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:16.488606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:16.488606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:16.488606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:16.489452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:16.489452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:16.489452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:16.489452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:16.489452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:16.489452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:16.489452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:16.489452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:16.489452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:16.489452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:16.489452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:14:16.489452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:16.489452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 184/367] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:14:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:16.490154 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8338df6-ee74-4d27-add1-be4480414a2b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:16.560959 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:16.564774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f504cd-509b-4e8a-a589-a5346e72d721 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:16.620493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8338df6-ee74-4d27-add1-be4480414a2b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:16.620791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422008f-105e-4b3b-83ad-5481085b21cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:16.621419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422008f-105e-4b3b-83ad-5481085b21cc None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:16.621704 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422008f-105e-4b3b-83ad-5481085b21cc None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:16.621981 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422008f-105e-4b3b-83ad-5481085b21cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:16.622713 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422008f-105e-4b3b-83ad-5481085b21cc None admin] RBAC: Adding query filter params (group.id=bb53a81c848a4c60968050d0c87a771d, role.id=5812e6538f3f4bdfbfbe11cd25c02957, scope.project.id=2269c4a7f991405497de468e3a9a745a) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:16.623081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422008f-105e-4b3b-83ad-5481085b21cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:16.624577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422008f-105e-4b3b-83ad-5481085b21cc None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:16.630679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 184/368] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 5 14:14:16 2022] GET /identity/v3/role_assignments?group.id=bb53a81c848a4c60968050d0c87a771d&scope.project.id=2269c4a7f991405497de468e3a9a745a&role.id=5812e6538f3f4bdfbfbe11cd25c02957 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:16.631602 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422008f-105e-4b3b-83ad-5481085b21cc None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:16.661555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79441a0-4761-4e20-8c90-38454afcf6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:16.662274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79441a0-4761-4e20-8c90-38454afcf6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:16.662559 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79441a0-4761-4e20-8c90-38454afcf6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:16.663485 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79441a0-4761-4e20-8c90-38454afcf6c8 None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:16.664973 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79441a0-4761-4e20-8c90-38454afcf6c8 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:16.666765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79441a0-4761-4e20-8c90-38454afcf6c8 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:16.668996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 185/369] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:16 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:16.669919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79441a0-4761-4e20-8c90-38454afcf6c8 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:16.741883 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:16.745163 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422008f-105e-4b3b-83ad-5481085b21cc None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:16.781575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09672002-b71a-4e13-8ff2-bd241ce7ebf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:16.782200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09672002-b71a-4e13-8ff2-bd241ce7ebf1 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:16.782484 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09672002-b71a-4e13-8ff2-bd241ce7ebf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:16.782762 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09672002-b71a-4e13-8ff2-bd241ce7ebf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:16.795479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79441a0-4761-4e20-8c90-38454afcf6c8 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:16.816177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:16.840073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2ce5b-0395-41a8-99b5-ddf0ae6864ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:16.841046 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2ce5b-0395-41a8-99b5-ddf0ae6864ec None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:16.841472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2ce5b-0395-41a8-99b5-ddf0ae6864ec None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:16.841887 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2ce5b-0395-41a8-99b5-ddf0ae6864ec None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:16.842769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2ce5b-0395-41a8-99b5-ddf0ae6864ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:16.843304 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2ce5b-0395-41a8-99b5-ddf0ae6864ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:16.844869 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2ce5b-0395-41a8-99b5-ddf0ae6864ec None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:16.856656 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 186/370] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:16.857753 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2ce5b-0395-41a8-99b5-ddf0ae6864ec None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:16.861341 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09672002-b71a-4e13-8ff2-bd241ce7ebf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:16.862874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09672002-b71a-4e13-8ff2-bd241ce7ebf1 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:16.966879 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 185/371] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:16.967900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09672002-b71a-4e13-8ff2-bd241ce7ebf1 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:16.976169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2ce5b-0395-41a8-99b5-ddf0ae6864ec None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:17.003279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:17.019263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a5abc-5edd-4724-ac65-780d37e80695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:17.020314 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a5abc-5edd-4724-ac65-780d37e80695 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:17.020766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a5abc-5edd-4724-ac65-780d37e80695 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:17.021178 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a5abc-5edd-4724-ac65-780d37e80695 None admin] PATH_INFO: `/v3/groups/bb53a81c848a4c60968050d0c87a771d` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:17.022634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a5abc-5edd-4724-ac65-780d37e80695 None admin] RBAC: Authorizing `identity:get_group(group_id=bb53a81c848a4c60968050d0c87a771d)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:17.024382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a5abc-5edd-4724-ac65-780d37e80695 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:17.027020 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 187/372] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 5 14:14:16 2022] GET /identity/v3/groups/bb53a81c848a4c60968050d0c87a771d => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:17.029225 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a5abc-5edd-4724-ac65-780d37e80695 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:17.036896 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:17.096972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09672002-b71a-4e13-8ff2-bd241ce7ebf1 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:17.146709 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2ff6c-ff91-4caf-960c-3a2f66c0279a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:17.147638 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2ff6c-ff91-4caf-960c-3a2f66c0279a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:17.147928 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2ff6c-ff91-4caf-960c-3a2f66c0279a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:17.148208 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2ff6c-ff91-4caf-960c-3a2f66c0279a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:17.167886 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a5abc-5edd-4724-ac65-780d37e80695 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:17.214464 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f965965-510d-4831-a947-0976d0cd932b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:17.215499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f965965-510d-4831-a947-0976d0cd932b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:17.216488 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f965965-510d-4831-a947-0976d0cd932b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:17.216965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f965965-510d-4831-a947-0976d0cd932b None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:17.218840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f965965-510d-4831-a947-0976d0cd932b None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:17.223169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f965965-510d-4831-a947-0976d0cd932b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:17.227146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 188/373] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:17 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:17.228310 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f965965-510d-4831-a947-0976d0cd932b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:17.233106 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2ff6c-ff91-4caf-960c-3a2f66c0279a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:17.235533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2ff6c-ff91-4caf-960c-3a2f66c0279a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:17.294130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:17.315542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:17.344820 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f965965-510d-4831-a947-0976d0cd932b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:17.376324 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 186/374] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:17.377729 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2ff6c-ff91-4caf-960c-3a2f66c0279a None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:17.385345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb0c63-465f-4c09-aa9e-4b11280b67ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:17.385964 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb0c63-465f-4c09-aa9e-4b11280b67ad None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:17.386246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb0c63-465f-4c09-aa9e-4b11280b67ad None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:17.386523 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb0c63-465f-4c09-aa9e-4b11280b67ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:17.387311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb0c63-465f-4c09-aa9e-4b11280b67ad None admin] RBAC: Adding query filter params (group.id=bb53a81c848a4c60968050d0c87a771d, role.id=5812e6538f3f4bdfbfbe11cd25c02957, scope.project.id=e5e3e090ce8348c19e0ddc4994094302) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:17.387681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb0c63-465f-4c09-aa9e-4b11280b67ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:17.389213 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb0c63-465f-4c09-aa9e-4b11280b67ad None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:17.395586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 189/375] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 5 14:14:17 2022] GET /identity/v3/role_assignments?group.id=bb53a81c848a4c60968050d0c87a771d&scope.project.id=e5e3e090ce8348c19e0ddc4994094302&role.id=5812e6538f3f4bdfbfbe11cd25c02957 => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:17.396374 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb0c63-465f-4c09-aa9e-4b11280b67ad None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:17.498357 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2ff6c-ff91-4caf-960c-3a2f66c0279a None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:17.508991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb0c63-465f-4c09-aa9e-4b11280b67ad None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:17.539977 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:17.546254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e9d3a-4394-41a4-b1d9-8611abaae26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:17.546864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e9d3a-4394-41a4-b1d9-8611abaae26d None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:17.547150 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e9d3a-4394-41a4-b1d9-8611abaae26d None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:17.547425 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e9d3a-4394-41a4-b1d9-8611abaae26d None admin] PATH_INFO: `/v3/projects/f189779a37444439ac0c4ef7b42cc427` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:17.548748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e9d3a-4394-41a4-b1d9-8611abaae26d None admin] RBAC: Authorizing `identity:get_project(project_id=f189779a37444439ac0c4ef7b42cc427)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:17.550317 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e9d3a-4394-41a4-b1d9-8611abaae26d None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:17.554829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 190/376] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:17 2022] GET /identity/v3/projects/f189779a37444439ac0c4ef7b42cc427 => generated 344 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:17.557766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e9d3a-4394-41a4-b1d9-8611abaae26d None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:17.581675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b1b26-860b-4940-b5ad-9d1a827bec54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:17.582663 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b1b26-860b-4940-b5ad-9d1a827bec54 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:17.583137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b1b26-860b-4940-b5ad-9d1a827bec54 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:17.583551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b1b26-860b-4940-b5ad-9d1a827bec54 None admin] PATH_INFO: `/v3/groups/bb53a81c848a4c60968050d0c87a771d` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:17.585890 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b1b26-860b-4940-b5ad-9d1a827bec54 None admin] RBAC: Authorizing `identity:get_group(group_id=bb53a81c848a4c60968050d0c87a771d)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:17.589592 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b1b26-860b-4940-b5ad-9d1a827bec54 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:17.593349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 187/377] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 5 14:14:17 2022] GET /identity/v3/groups/bb53a81c848a4c60968050d0c87a771d => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:17.599265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b1b26-860b-4940-b5ad-9d1a827bec54 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:17.654862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:17.696416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e9d3a-4394-41a4-b1d9-8611abaae26d None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:17.736236 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a7c35-df35-4de5-a1af-3a7951f3dc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:17.737236 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a7c35-df35-4de5-a1af-3a7951f3dc42 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:17.737719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b1b26-860b-4940-b5ad-9d1a827bec54 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:17.737859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a7c35-df35-4de5-a1af-3a7951f3dc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:17.738329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a7c35-df35-4de5-a1af-3a7951f3dc42 None admin] PATH_INFO: `/v3/groups/afab53ef69d74dc7b83e006ec792f927` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:17.743672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a7c35-df35-4de5-a1af-3a7951f3dc42 None admin] RBAC: Authorizing `identity:get_group(group_id=afab53ef69d74dc7b83e006ec792f927)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:17.745432 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a7c35-df35-4de5-a1af-3a7951f3dc42 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:17.747820 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 191/378] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 5 14:14:17 2022] GET /identity/v3/groups/afab53ef69d74dc7b83e006ec792f927 => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:17.756970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a7c35-df35-4de5-a1af-3a7951f3dc42 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:17.808187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10ebfe-3cad-4889-8cb8-7fe1520efa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:17.809141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10ebfe-3cad-4889-8cb8-7fe1520efa37 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:17.809620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10ebfe-3cad-4889-8cb8-7fe1520efa37 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:17.811667 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10ebfe-3cad-4889-8cb8-7fe1520efa37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:17.812607 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10ebfe-3cad-4889-8cb8-7fe1520efa37 None admin] RBAC: Adding query filter params (group.id=bb53a81c848a4c60968050d0c87a771d, role.id=1182ffc3e9f9477db2003500b7bfd402, scope.project.id=e5e3e090ce8348c19e0ddc4994094302) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:17.813882 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10ebfe-3cad-4889-8cb8-7fe1520efa37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:17.817238 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10ebfe-3cad-4889-8cb8-7fe1520efa37 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:17.830758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 188/379] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 5 14:14:17 2022] GET /identity/v3/role_assignments?group.id=bb53a81c848a4c60968050d0c87a771d&scope.project.id=e5e3e090ce8348c19e0ddc4994094302&role.id=1182ffc3e9f9477db2003500b7bfd402 => generated 260 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:17.873180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:17.900542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a7c35-df35-4de5-a1af-3a7951f3dc42 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:17.951593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6543c10-bafa-4f81-9668-817fe8c980a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:17.952239 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6543c10-bafa-4f81-9668-817fe8c980a1 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:17.952526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6543c10-bafa-4f81-9668-817fe8c980a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:17.952808 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6543c10-bafa-4f81-9668-817fe8c980a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:17.953645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6543c10-bafa-4f81-9668-817fe8c980a1 None admin] RBAC: Adding query filter params (group.id=afab53ef69d74dc7b83e006ec792f927, role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=f189779a37444439ac0c4ef7b42cc427) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:17.953958 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6543c10-bafa-4f81-9668-817fe8c980a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:17.955533 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6543c10-bafa-4f81-9668-817fe8c980a1 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:17.962002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 192/380] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 5 14:14:17 2022] GET /identity/v3/role_assignments?group.id=afab53ef69d74dc7b83e006ec792f927&scope.project.id=f189779a37444439ac0c4ef7b42cc427&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:18.644679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be311e23-31bc-4aaf-b8fa-0e1f1e20ade6 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:18.645191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be311e23-31bc-4aaf-b8fa-0e1f1e20ade6 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:18.645264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be311e23-31bc-4aaf-b8fa-0e1f1e20ade6 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:18.646836 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 189/381] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:18.652165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f60f00-1b63-4b47-ba2e-6d5c6d523a41 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:18.652588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f60f00-1b63-4b47-ba2e-6d5c6d523a41 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:18.652756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f60f00-1b63-4b47-ba2e-6d5c6d523a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:18.679834 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-71f60f00-1b63-4b47-ba2e-6d5c6d523a41 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:18.681069 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-71f60f00-1b63-4b47-ba2e-6d5c6d523a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:18.769585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 193/382] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:18.947983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef675e56-8def-43a9-b304-d5b90bb14e57 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:18.948306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef675e56-8def-43a9-b304-d5b90bb14e57 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:18.948480 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef675e56-8def-43a9-b304-d5b90bb14e57 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:18.949791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 190/383] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:18.955063 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb836b9-9367-4db6-abec-9ba3a7bb9286 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:18.955352 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb836b9-9367-4db6-abec-9ba3a7bb9286 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:18.955626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb836b9-9367-4db6-abec-9ba3a7bb9286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:18.981522 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-6bb836b9-9367-4db6-abec-9ba3a7bb9286 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:18.982767 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-6bb836b9-9367-4db6-abec-9ba3a7bb9286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:19.030904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:19.060585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 194/384] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:19.196215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef675e56-8def-43a9-b304-d5b90bb14e57 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:19.325082 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:19.332650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef675e56-8def-43a9-b304-d5b90bb14e57 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:19.384042 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3bd29d-20bb-48dd-a5ab-aeca06541d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:19.384711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3bd29d-20bb-48dd-a5ab-aeca06541d6f None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:19.384999 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3bd29d-20bb-48dd-a5ab-aeca06541d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:19.385274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3bd29d-20bb-48dd-a5ab-aeca06541d6f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:19.402169 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3bd29d-20bb-48dd-a5ab-aeca06541d6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:19.407311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-6e3bd29d-20bb-48dd-a5ab-aeca06541d6f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:14:19.407311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:19.407311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:19.407311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:19.407311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:19.407311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:19.407311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:19.407311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:19.407311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:19.407311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:19.407311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:19.407311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:19.407311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:19.407311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:19.407311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:19.407311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:19.408252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:19.408252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:19.408252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:19.408252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:19.408252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:19.408252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:19.408252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:19.408252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:19.408252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:19.408252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:19.408252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:19.408252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:19.408252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:19.408252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:19.408252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:19.408252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:19.409052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:19.409052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:19.409052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:19.409052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:19.409052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:19.409052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:19.409052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:19.409052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:19.409052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:19.409052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:19.409052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:14:19.409052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:19.409052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 191/385] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:14:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:19.414245 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb836b9-9367-4db6-abec-9ba3a7bb9286 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:19.445482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3bd29d-20bb-48dd-a5ab-aeca06541d6f None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:19.614531 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb836b9-9367-4db6-abec-9ba3a7bb9286 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:19.633589 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3bd29d-20bb-48dd-a5ab-aeca06541d6f None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:19.683172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c5ead-f642-47ac-894b-92066cbed5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:19.684465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c5ead-f642-47ac-894b-92066cbed5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:19.684958 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c5ead-f642-47ac-894b-92066cbed5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:19.685422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c5ead-f642-47ac-894b-92066cbed5a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:19.686369 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c5ead-f642-47ac-894b-92066cbed5a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:19.688321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c5ead-f642-47ac-894b-92066cbed5a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:19.689980 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c5ead-f642-47ac-894b-92066cbed5a6 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:19.706368 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8782032-0203-472f-89b3-b408019f9295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:19.708675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8782032-0203-472f-89b3-b408019f9295 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:19.709175 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8782032-0203-472f-89b3-b408019f9295 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:19.710681 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8782032-0203-472f-89b3-b408019f9295 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:19.711614 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 195/386] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:19.718554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c5ead-f642-47ac-894b-92066cbed5a6 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:19.734162 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8782032-0203-472f-89b3-b408019f9295 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:19.746547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-a8782032-0203-472f-89b3-b408019f9295 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:19.746547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:19.746547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:19.746547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:19.746547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:19.746547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:19.746547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:19.746547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:19.746547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:19.746547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:19.746547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:19.746547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:19.746547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:19.746547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:19.746547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:19.746547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:19.747482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:19.747482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:19.747482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:19.747482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:19.747482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:19.747482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:19.747482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:19.747482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:19.747482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:19.747482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:19.747482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:19.747482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:19.747482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:19.747482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:19.747482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:19.747482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:19.748275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:19.748275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:19.748275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:19.748275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:19.748275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:19.748275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:19.748275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:19.748275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:19.748275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:19.748275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:19.748275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:19.748275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:19.750938 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 192/387] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:14:19.756813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8782032-0203-472f-89b3-b408019f9295 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:19.764964 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:19.782830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:19.836924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c5ead-f642-47ac-894b-92066cbed5a6 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:19.873197 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bddf15-9ebc-48f8-87d2-aba1200fc29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:19.873751 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bddf15-9ebc-48f8-87d2-aba1200fc29a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:19.874032 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bddf15-9ebc-48f8-87d2-aba1200fc29a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:19.874306 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bddf15-9ebc-48f8-87d2-aba1200fc29a None admin] PATH_INFO: `/v3/groups/bb53a81c848a4c60968050d0c87a771d` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:19.875459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bddf15-9ebc-48f8-87d2-aba1200fc29a None admin] RBAC: Authorizing `identity:get_group(group_id=bb53a81c848a4c60968050d0c87a771d)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:19.876895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bddf15-9ebc-48f8-87d2-aba1200fc29a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:19.878743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 196/388] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 5 14:14:19 2022] GET /identity/v3/groups/bb53a81c848a4c60968050d0c87a771d => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:19.884926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd35bc0-35dd-470b-9196-9eafc6dfabe2 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:19.885217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd35bc0-35dd-470b-9196-9eafc6dfabe2 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:19.885493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd35bc0-35dd-470b-9196-9eafc6dfabe2 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:19.886857 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8782032-0203-472f-89b3-b408019f9295 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:19.886969 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 197/389] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:19.887681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd35bc0-35dd-470b-9196-9eafc6dfabe2 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:19.926055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23246b3-cf62-4839-bf96-898a66ebe6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:19.926729 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23246b3-cf62-4839-bf96-898a66ebe6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:19.927041 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23246b3-cf62-4839-bf96-898a66ebe6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:19.927315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23246b3-cf62-4839-bf96-898a66ebe6d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:19.928033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23246b3-cf62-4839-bf96-898a66ebe6d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:19.928334 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23246b3-cf62-4839-bf96-898a66ebe6d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:19.929768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23246b3-cf62-4839-bf96-898a66ebe6d3 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:19.938765 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 193/390] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:14:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:19.940542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebe411-f77e-4f2c-b018-a8b96760fd98 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:19.940921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebe411-f77e-4f2c-b018-a8b96760fd98 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:19.941322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebe411-f77e-4f2c-b018-a8b96760fd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:19.960243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-33ebe411-f77e-4f2c-b018-a8b96760fd98 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:19.961588 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-33ebe411-f77e-4f2c-b018-a8b96760fd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:20.015233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd35bc0-35dd-470b-9196-9eafc6dfabe2 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:20.024360 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:20.032752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 194/391] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:20.034012 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebe411-f77e-4f2c-b018-a8b96760fd98 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:20.063162 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:20.083643 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689372c0-ffd2-4589-87af-8e144728b489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:20.084648 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689372c0-ffd2-4589-87af-8e144728b489 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:20.085228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689372c0-ffd2-4589-87af-8e144728b489 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:20.085695 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689372c0-ffd2-4589-87af-8e144728b489 None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:20.087311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689372c0-ffd2-4589-87af-8e144728b489 None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:20.089767 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689372c0-ffd2-4589-87af-8e144728b489 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:20.094104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 198/392] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:19 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:20.100488 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689372c0-ffd2-4589-87af-8e144728b489 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:20.230155 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689372c0-ffd2-4589-87af-8e144728b489 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:20.234028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebe411-f77e-4f2c-b018-a8b96760fd98 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:20.269208 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765d1cc-e4b7-41aa-ae6b-987d01abb3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:20.270342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765d1cc-e4b7-41aa-ae6b-987d01abb3a3 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:20.270764 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765d1cc-e4b7-41aa-ae6b-987d01abb3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:20.271219 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765d1cc-e4b7-41aa-ae6b-987d01abb3a3 None admin] PATH_INFO: `/v3/groups/bb53a81c848a4c60968050d0c87a771d` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:20.271379 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3de3a-954d-4c7d-9b62-14b1f2811d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:20.272159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3de3a-954d-4c7d-9b62-14b1f2811d70 None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:20.272536 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3de3a-954d-4c7d-9b62-14b1f2811d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:20.272905 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3de3a-954d-4c7d-9b62-14b1f2811d70 None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a/groups/bb53a81c848a4c60968050d0c87a771d/roles/1182ffc3e9f9477db2003500b7bfd402` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:20.273118 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765d1cc-e4b7-41aa-ae6b-987d01abb3a3 None admin] RBAC: Authorizing `identity:get_group(group_id=bb53a81c848a4c60968050d0c87a771d)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:20.274739 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765d1cc-e4b7-41aa-ae6b-987d01abb3a3 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:20.275912 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3de3a-954d-4c7d-9b62-14b1f2811d70 None admin] RBAC: Authorizing `identity:create_grant(project_id=2269c4a7f991405497de468e3a9a745a, group_id=bb53a81c848a4c60968050d0c87a771d, role_id=1182ffc3e9f9477db2003500b7bfd402)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:20.281426 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3de3a-954d-4c7d-9b62-14b1f2811d70 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:20.281959 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 195/393] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 5 14:14:20 2022] GET /identity/v3/groups/bb53a81c848a4c60968050d0c87a771d => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:20.283364 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed69911-e3d5-43cc-ab6c-80158fb0d562 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:20.283641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed69911-e3d5-43cc-ab6c-80158fb0d562 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:20.283910 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed69911-e3d5-43cc-ab6c-80158fb0d562 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:20.285250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 196/394] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:20.285801 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed69911-e3d5-43cc-ab6c-80158fb0d562 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:20.325514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 199/395] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Sep 5 14:14:20 2022] PUT /identity/v3/projects/2269c4a7f991405497de468e3a9a745a/groups/bb53a81c848a4c60968050d0c87a771d/roles/1182ffc3e9f9477db2003500b7bfd402 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:14:20.327172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e43741-87a9-4958-b452-648c65dbc940 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:20.327635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e43741-87a9-4958-b452-648c65dbc940 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:20.328082 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e43741-87a9-4958-b452-648c65dbc940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:20.349724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-70e43741-87a9-4958-b452-648c65dbc940 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:20.351026 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-70e43741-87a9-4958-b452-648c65dbc940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:20.371929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:20.382737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:20.401575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed69911-e3d5-43cc-ab6c-80158fb0d562 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:20.432448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 200/396] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:20.433876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b445a-1b09-41d6-a987-ddc2717045e1 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:20.434240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b445a-1b09-41d6-a987-ddc2717045e1 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:20.434583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b445a-1b09-41d6-a987-ddc2717045e1 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:20.438850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 201/397] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:20.441425 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b445a-1b09-41d6-a987-ddc2717045e1 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:20.450544 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919bc98-7f57-4c01-9830-b2406180e62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:20.451585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919bc98-7f57-4c01-9830-b2406180e62f None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:20.451938 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919bc98-7f57-4c01-9830-b2406180e62f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:20.464320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919bc98-7f57-4c01-9830-b2406180e62f None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:20.464320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919bc98-7f57-4c01-9830-b2406180e62f None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:20.464320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919bc98-7f57-4c01-9830-b2406180e62f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:20.464320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 197/398] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:20 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:20.464320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81a9db-e140-488d-b256-082127e026c6 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:20.464320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81a9db-e140-488d-b256-082127e026c6 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:20.464736 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81a9db-e140-488d-b256-082127e026c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:20.479614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-2e81a9db-e140-488d-b256-082127e026c6 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:20.480576 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-2e81a9db-e140-488d-b256-082127e026c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:20.562814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 198/399] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:20.562814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81a9db-e140-488d-b256-082127e026c6 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:20.613689 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b445a-1b09-41d6-a987-ddc2717045e1 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:20.676839 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316835eb-28ad-4d85-8af4-84facc65511e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:20.677542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316835eb-28ad-4d85-8af4-84facc65511e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:20.677830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316835eb-28ad-4d85-8af4-84facc65511e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:20.678106 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316835eb-28ad-4d85-8af4-84facc65511e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:20.684783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:20.701198 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316835eb-28ad-4d85-8af4-84facc65511e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:20.709300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-316835eb-28ad-4d85-8af4-84facc65511e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:20.709300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:20.709300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:20.709300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:20.709300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:20.709300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:20.709300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:20.709300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:20.709300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:20.709300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:20.709300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:20.709300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:20.709300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:20.709300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:20.709300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:20.709300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:20.710484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:20.710484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:20.710484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:20.710484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:20.710484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:20.710484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:20.710484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:20.710484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:20.710484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:20.710484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:20.710484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:20.710484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:20.710484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:20.710484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:20.710484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:20.710484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:20.711666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:20.711666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:20.711666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:20.711666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:20.711666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:20.711666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:20.711666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:20.711666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:20.711666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:20.711666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:20.711666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:20.711666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:20.711666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 202/400] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:20.715966 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316835eb-28ad-4d85-8af4-84facc65511e None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:20.734356 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81a9db-e140-488d-b256-082127e026c6 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:20.742939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:20.770070 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa95c3-d86e-4389-bbce-373cb6606458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:20.770843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa95c3-d86e-4389-bbce-373cb6606458 None admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:20.771132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa95c3-d86e-4389-bbce-373cb6606458 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:20.771406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa95c3-d86e-4389-bbce-373cb6606458 None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a/groups/bb53a81c848a4c60968050d0c87a771d/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:20.773588 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa95c3-d86e-4389-bbce-373cb6606458 None admin] RBAC: Authorizing `identity:create_grant(project_id=2269c4a7f991405497de468e3a9a745a, group_id=bb53a81c848a4c60968050d0c87a771d, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:20.775424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa95c3-d86e-4389-bbce-373cb6606458 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:20.830822 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316835eb-28ad-4d85-8af4-84facc65511e None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:20.870839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 199/401] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Sep 5 14:14:20 2022] PUT /identity/v3/projects/2269c4a7f991405497de468e3a9a745a/groups/bb53a81c848a4c60968050d0c87a771d/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:14:20.870839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa95c3-d86e-4389-bbce-373cb6606458 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:20.936037 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5c160-987d-4364-ab5c-11b27e400699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:20.937981 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5c160-987d-4364-ab5c-11b27e400699 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:20.938697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5c160-987d-4364-ab5c-11b27e400699 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:20.939201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5c160-987d-4364-ab5c-11b27e400699 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:20.964417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f5c160-987d-4364-ab5c-11b27e400699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:20.969995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-97f5c160-987d-4364-ab5c-11b27e400699 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:14:20.969995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:20.969995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:20.969995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:20.969995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:20.969995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:20.969995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:20.969995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:20.969995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:20.969995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:20.969995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:20.969995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:20.969995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:20.969995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:20.969995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:20.969995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:20.971022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:20.971022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:20.971022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:20.971022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:20.971022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:20.971022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:20.971022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:20.971022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:20.971022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:20.971022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:20.971022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:20.971022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:20.971022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:20.971022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:20.971022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:20.971022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:20.971838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:20.971838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:20.971838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:20.971838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:20.971838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:20.971838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:20.971838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:20.971838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:20.971838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:20.971838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:20.971838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:14:20.971838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:20.974196 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 203/402] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:14:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:14:20.975424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5c160-987d-4364-ab5c-11b27e400699 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:21.078640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa95c3-d86e-4389-bbce-373cb6606458 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:21.112315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:21.122068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:21.128543 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db4af5-2ef2-4560-bfd3-909694d14661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:21.129442 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db4af5-2ef2-4560-bfd3-909694d14661 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:21.129734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db4af5-2ef2-4560-bfd3-909694d14661 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:21.130010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db4af5-2ef2-4560-bfd3-909694d14661 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:21.130678 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db4af5-2ef2-4560-bfd3-909694d14661 None admin] RBAC: Adding query filter params (name=member) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:21.131003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db4af5-2ef2-4560-bfd3-909694d14661 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:21.132561 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db4af5-2ef2-4560-bfd3-909694d14661 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:21.136779 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5c160-987d-4364-ab5c-11b27e400699 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:21.143365 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 200/403] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:14:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:21.144200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db4af5-2ef2-4560-bfd3-909694d14661 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:21.199564 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398b6aa-a2fe-43e8-8642-2c1582613394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:21.200922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398b6aa-a2fe-43e8-8642-2c1582613394 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:21.201721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398b6aa-a2fe-43e8-8642-2c1582613394 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:21.202212 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398b6aa-a2fe-43e8-8642-2c1582613394 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:21.242221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398b6aa-a2fe-43e8-8642-2c1582613394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:21.253829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-b398b6aa-a2fe-43e8-8642-2c1582613394 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:14:21.253829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:21.253829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:21.253829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:21.253829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:21.253829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:21.253829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:21.253829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:21.253829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:21.253829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:21.253829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:21.253829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:21.253829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:21.253829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:21.253829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:21.253829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:21.254890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:21.254890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:21.254890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:21.254890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:21.254890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:21.254890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:21.254890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:21.254890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:21.254890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:21.254890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:21.254890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:21.254890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:21.254890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:21.254890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:21.254890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:21.254890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:21.255778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:21.255778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:21.255778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:21.255778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:21.255778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:21.255778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:21.255778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:21.255778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:21.255778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:21.255778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:21.255778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:14:21.255778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:21.260909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 204/404] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:14:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:21.262142 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398b6aa-a2fe-43e8-8642-2c1582613394 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:21.344266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db4af5-2ef2-4560-bfd3-909694d14661 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:21.376524 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398b6aa-a2fe-43e8-8642-2c1582613394 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:21.423783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88aac0-be38-4e0d-937b-1551a56b91ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:21.424434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88aac0-be38-4e0d-937b-1551a56b91ad None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:21.424718 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88aac0-be38-4e0d-937b-1551a56b91ad None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:21.424996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88aac0-be38-4e0d-937b-1551a56b91ad None admin] PATH_INFO: `/v3/groups/bb53a81c848a4c60968050d0c87a771d` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:21.425567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3a580-1e62-4f52-990a-eede9ab674ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:21.426136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88aac0-be38-4e0d-937b-1551a56b91ad None admin] RBAC: Authorizing `identity:get_group(group_id=bb53a81c848a4c60968050d0c87a771d)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:21.426630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3a580-1e62-4f52-990a-eede9ab674ca None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:21.427131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3a580-1e62-4f52-990a-eede9ab674ca None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:21.427606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3a580-1e62-4f52-990a-eede9ab674ca None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:21.427714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88aac0-be38-4e0d-937b-1551a56b91ad None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:21.428575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f3a580-1e62-4f52-990a-eede9ab674ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:21.429128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f3a580-1e62-4f52-990a-eede9ab674ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:21.429491 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 205/405] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 5 14:14:21 2022] GET /identity/v3/groups/bb53a81c848a4c60968050d0c87a771d => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:21.430331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88aac0-be38-4e0d-937b-1551a56b91ad None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:21.431558 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f3a580-1e62-4f52-990a-eede9ab674ca None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:21.441247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 201/406] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:21.442521 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3a580-1e62-4f52-990a-eede9ab674ca None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:21.447792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:21.501729 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:21.540333 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88aac0-be38-4e0d-937b-1551a56b91ad None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:21.549338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3a580-1e62-4f52-990a-eede9ab674ca None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:21.578440 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf75d8-66f7-4619-b357-e816cb4b45e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:21.579458 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf75d8-66f7-4619-b357-e816cb4b45e3 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:21.579948 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf75d8-66f7-4619-b357-e816cb4b45e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:21.580506 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf75d8-66f7-4619-b357-e816cb4b45e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:21.581346 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf75d8-66f7-4619-b357-e816cb4b45e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:21.581837 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf75d8-66f7-4619-b357-e816cb4b45e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:21.583466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf75d8-66f7-4619-b357-e816cb4b45e3 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:21.585230 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ee890-6d77-4d30-bd6d-5f09a2b93aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:21.585869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ee890-6d77-4d30-bd6d-5f09a2b93aec None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:21.586371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ee890-6d77-4d30-bd6d-5f09a2b93aec None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:21.586494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ee890-6d77-4d30-bd6d-5f09a2b93aec None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:21.587769 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ee890-6d77-4d30-bd6d-5f09a2b93aec None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:21.589230 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ee890-6d77-4d30-bd6d-5f09a2b93aec None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:21.591371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 202/407] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:21 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:21.592215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ee890-6d77-4d30-bd6d-5f09a2b93aec None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:21.598411 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 206/408] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:21.599511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf75d8-66f7-4619-b357-e816cb4b45e3 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:21.678154 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:21.697430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ee890-6d77-4d30-bd6d-5f09a2b93aec None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:21.705190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf75d8-66f7-4619-b357-e816cb4b45e3 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:21.726059 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:21.737592 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3390fbfb-9737-4008-aedb-c7313efafb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:21.738303 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390fbfb-9737-4008-aedb-c7313efafb35 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:21.738515 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390fbfb-9737-4008-aedb-c7313efafb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:21.738823 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390fbfb-9737-4008-aedb-c7313efafb35 None admin] PATH_INFO: `/v3/groups/bb53a81c848a4c60968050d0c87a771d` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:21.740034 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3390fbfb-9737-4008-aedb-c7313efafb35 None admin] RBAC: Authorizing `identity:get_group(group_id=bb53a81c848a4c60968050d0c87a771d)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:21.741603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3390fbfb-9737-4008-aedb-c7313efafb35 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:21.743966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 203/409] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 5 14:14:21 2022] GET /identity/v3/groups/bb53a81c848a4c60968050d0c87a771d => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:21.751267 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3390fbfb-9737-4008-aedb-c7313efafb35 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:21.751385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff183ed-c9af-4da9-a719-589853db57f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:21.751385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff183ed-c9af-4da9-a719-589853db57f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:21.751385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff183ed-c9af-4da9-a719-589853db57f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:21.751385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff183ed-c9af-4da9-a719-589853db57f3 None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302/groups/bb53a81c848a4c60968050d0c87a771d/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:21.751681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff183ed-c9af-4da9-a719-589853db57f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5e3e090ce8348c19e0ddc4994094302, group_id=bb53a81c848a4c60968050d0c87a771d, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:21.751681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff183ed-c9af-4da9-a719-589853db57f3 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:21.848857 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3390fbfb-9737-4008-aedb-c7313efafb35 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:21.859118 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 207/410] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Sep 5 14:14:21 2022] PUT /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302/groups/bb53a81c848a4c60968050d0c87a771d/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:14:21.866278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff183ed-c9af-4da9-a719-589853db57f3 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:21.891198 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a313be-7a22-48ab-9b1f-2b96340fa7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:21.891198 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a313be-7a22-48ab-9b1f-2b96340fa7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:21.891198 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a313be-7a22-48ab-9b1f-2b96340fa7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:21.891198 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a313be-7a22-48ab-9b1f-2b96340fa7f3 None admin] PATH_INFO: `/v3/groups/afab53ef69d74dc7b83e006ec792f927` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:21.891635 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a313be-7a22-48ab-9b1f-2b96340fa7f3 None admin] RBAC: Authorizing `identity:get_group(group_id=afab53ef69d74dc7b83e006ec792f927)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:21.891635 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a313be-7a22-48ab-9b1f-2b96340fa7f3 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:21.892381 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 204/411] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 5 14:14:21 2022] GET /identity/v3/groups/afab53ef69d74dc7b83e006ec792f927 => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:21.901627 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a313be-7a22-48ab-9b1f-2b96340fa7f3 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:21.995294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:22.021744 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff183ed-c9af-4da9-a719-589853db57f3 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:22.043408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a313be-7a22-48ab-9b1f-2b96340fa7f3 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:22.082755 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ae36c-10dc-4b23-b713-fdb12c391ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:22.083438 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ae36c-10dc-4b23-b713-fdb12c391ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:22.083785 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ae36c-10dc-4b23-b713-fdb12c391ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:22.084065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ae36c-10dc-4b23-b713-fdb12c391ed3 None admin] PATH_INFO: `/v3/projects/f189779a37444439ac0c4ef7b42cc427` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:22.085215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ae36c-10dc-4b23-b713-fdb12c391ed3 None admin] RBAC: Authorizing `identity:get_project(project_id=f189779a37444439ac0c4ef7b42cc427)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:22.086685 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ae36c-10dc-4b23-b713-fdb12c391ed3 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:22.092545 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 205/412] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:21 2022] GET /identity/v3/projects/f189779a37444439ac0c4ef7b42cc427 => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:22.094928 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ae36c-10dc-4b23-b713-fdb12c391ed3 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:22.121064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b7a32-98f1-4a6e-bd76-bb40f0c7e447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:22.122169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b7a32-98f1-4a6e-bd76-bb40f0c7e447 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:22.122757 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b7a32-98f1-4a6e-bd76-bb40f0c7e447 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:22.123435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b7a32-98f1-4a6e-bd76-bb40f0c7e447 None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:22.125402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b7a32-98f1-4a6e-bd76-bb40f0c7e447 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:22.127753 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b7a32-98f1-4a6e-bd76-bb40f0c7e447 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:22.130133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 208/413] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:21 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:22.143792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b7a32-98f1-4a6e-bd76-bb40f0c7e447 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:22.148493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:22.239781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ae36c-10dc-4b23-b713-fdb12c391ed3 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:22.291360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:22.305840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cbecb-2af6-4a11-aa70-e279f7a747bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:22.306569 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cbecb-2af6-4a11-aa70-e279f7a747bb None admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:22.306911 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cbecb-2af6-4a11-aa70-e279f7a747bb None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:22.307193 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cbecb-2af6-4a11-aa70-e279f7a747bb None admin] PATH_INFO: `/v3/projects/f189779a37444439ac0c4ef7b42cc427/groups/afab53ef69d74dc7b83e006ec792f927/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:22.311007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70cbecb-2af6-4a11-aa70-e279f7a747bb None admin] RBAC: Authorizing `identity:create_grant(project_id=f189779a37444439ac0c4ef7b42cc427, group_id=afab53ef69d74dc7b83e006ec792f927, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:22.313060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70cbecb-2af6-4a11-aa70-e279f7a747bb None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:22.317361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b7a32-98f1-4a6e-bd76-bb40f0c7e447 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:22.355263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fd864-31e1-4c3c-a3b5-0e06e0d3a4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:22.355936 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fd864-31e1-4c3c-a3b5-0e06e0d3a4c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:22.356275 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fd864-31e1-4c3c-a3b5-0e06e0d3a4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:22.356557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fd864-31e1-4c3c-a3b5-0e06e0d3a4c1 None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302/groups/bb53a81c848a4c60968050d0c87a771d/roles/1182ffc3e9f9477db2003500b7bfd402` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:22.359039 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fd864-31e1-4c3c-a3b5-0e06e0d3a4c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5e3e090ce8348c19e0ddc4994094302, group_id=bb53a81c848a4c60968050d0c87a771d, role_id=1182ffc3e9f9477db2003500b7bfd402)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:22.360763 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fd864-31e1-4c3c-a3b5-0e06e0d3a4c1 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:22.427698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 206/414] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Sep 5 14:14:22 2022] PUT /identity/v3/projects/f189779a37444439ac0c4ef7b42cc427/groups/afab53ef69d74dc7b83e006ec792f927/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:14:22.428912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6ac09-a16b-4e70-ad8d-5a8544abe958 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:22.429203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6ac09-a16b-4e70-ad8d-5a8544abe958 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:22.429477 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6ac09-a16b-4e70-ad8d-5a8544abe958 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:22.430985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 207/415] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:22.441149 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9c452-f963-444e-8e2e-f5532d105b39 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:22.441500 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9c452-f963-444e-8e2e-f5532d105b39 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:22.441712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9c452-f963-444e-8e2e-f5532d105b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:22.463301 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 209/416] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Sep 5 14:14:22 2022] PUT /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302/groups/bb53a81c848a4c60968050d0c87a771d/roles/1182ffc3e9f9477db2003500b7bfd402 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:14:22.471185 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-d0e9c452-f963-444e-8e2e-f5532d105b39 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:22.472586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d0e9c452-f963-444e-8e2e-f5532d105b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:22.579955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 208/417] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:14:22.871174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fd864-31e1-4c3c-a3b5-0e06e0d3a4c1 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:22.951214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:22.999052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fd864-31e1-4c3c-a3b5-0e06e0d3a4c1 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:23.001428 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e6af7-ee18-446c-97cb-709ec22e38bd None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:23.001874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e6af7-ee18-446c-97cb-709ec22e38bd None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:23.002046 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e6af7-ee18-446c-97cb-709ec22e38bd None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:23.003702 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 209/418] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:23.012070 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bad3b-3273-4eb3-afe6-ebba4048138e None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:23.012360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bad3b-3273-4eb3-afe6-ebba4048138e None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:23.012641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bad3b-3273-4eb3-afe6-ebba4048138e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:23.040315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-190bad3b-3273-4eb3-afe6-ebba4048138e None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:23.042109 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-190bad3b-3273-4eb3-afe6-ebba4048138e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:23.049511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebad4c-bbf7-46ca-ae9f-f0d9ff28e9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:23.050266 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebad4c-bbf7-46ca-ae9f-f0d9ff28e9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:23.050553 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebad4c-bbf7-46ca-ae9f-f0d9ff28e9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:23.050847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebad4c-bbf7-46ca-ae9f-f0d9ff28e9f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:23.066756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebad4c-bbf7-46ca-ae9f-f0d9ff28e9f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:23.085707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-1aebad4c-bbf7-46ca-ae9f-f0d9ff28e9f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:14:23.085707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:23.085707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:23.085707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:23.085707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:23.085707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:23.085707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:23.085707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:23.085707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:23.085707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:23.085707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:23.085707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:23.085707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:23.085707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:23.085707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:23.085707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:23.086645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:23.086645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:23.086645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:23.086645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:23.086645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:23.086645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:23.086645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:23.086645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:23.086645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:23.086645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:23.086645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:23.086645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:23.086645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:23.086645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:23.086645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:23.086645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:23.087664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:23.087664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:23.087664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:23.087664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:23.087664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:23.087664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:23.087664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:23.087664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:23.087664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:23.087664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:23.087664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:14:23.087664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:23.087664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 210/419] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:14:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:23.095111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebad4c-bbf7-46ca-ae9f-f0d9ff28e9f3 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:23.123137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 210/420] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:23.263325 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebad4c-bbf7-46ca-ae9f-f0d9ff28e9f3 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:23.294611 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:23.311690 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a1146-d10f-4060-b2bb-c7d7f2371311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:23.312309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a1146-d10f-4060-b2bb-c7d7f2371311 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:23.312587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a1146-d10f-4060-b2bb-c7d7f2371311 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:23.312860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a1146-d10f-4060-b2bb-c7d7f2371311 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:23.313535 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a1146-d10f-4060-b2bb-c7d7f2371311 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:23.313838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a1146-d10f-4060-b2bb-c7d7f2371311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:23.315502 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a1146-d10f-4060-b2bb-c7d7f2371311 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:23.328942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 211/421] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:23.330296 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190bad3b-3273-4eb3-afe6-ebba4048138e None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:23.388835 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:23.461462 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190bad3b-3273-4eb3-afe6-ebba4048138e None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:23.506055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c4b88-ab7c-498a-acf7-f0cbb99929bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:23.506675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c4b88-ab7c-498a-acf7-f0cbb99929bc None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:23.506984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c4b88-ab7c-498a-acf7-f0cbb99929bc None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:23.507260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c4b88-ab7c-498a-acf7-f0cbb99929bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:23.521621 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a1146-d10f-4060-b2bb-c7d7f2371311 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:23.585496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c4b88-ab7c-498a-acf7-f0cbb99929bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:23.587086 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c4b88-ab7c-498a-acf7-f0cbb99929bc None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:23.654923 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a1146-d10f-4060-b2bb-c7d7f2371311 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:23.665316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:23.691320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344e07c-012b-4e40-8235-da8bebd9de76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:23.692212 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344e07c-012b-4e40-8235-da8bebd9de76 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:23.692640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344e07c-012b-4e40-8235-da8bebd9de76 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:23.693064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344e07c-012b-4e40-8235-da8bebd9de76 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:23.704235 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344e07c-012b-4e40-8235-da8bebd9de76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:23.709326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-4344e07c-012b-4e40-8235-da8bebd9de76 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:23.709326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:23.709326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:23.709326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:23.709326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:23.709326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:23.709326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:23.709326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:23.709326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:23.709326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:23.709326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:23.709326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:23.709326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:23.709326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:23.709326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:23.709326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:23.710240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:23.710240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:23.710240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:23.710240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:23.710240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:23.710240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:23.710240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:23.710240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:23.710240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:23.710240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:23.710240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:23.710240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:23.710240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:23.710240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:23.710240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:23.710240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:23.711083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:23.711083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:23.711083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:23.711083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:23.711083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:23.711083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:23.711083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:23.711083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:23.711083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:23.711083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:23.711083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:23.711083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:23.717832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 212/422] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:23.719257 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344e07c-012b-4e40-8235-da8bebd9de76 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:23.761984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 211/423] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:23.767342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c4b88-ab7c-498a-acf7-f0cbb99929bc None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:23.773942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:23.827993 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344e07c-012b-4e40-8235-da8bebd9de76 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:23.864075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b502c-5d14-4af6-9784-695b1a26f024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:23.865059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b502c-5d14-4af6-9784-695b1a26f024 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:23.865550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b502c-5d14-4af6-9784-695b1a26f024 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:23.866025 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b502c-5d14-4af6-9784-695b1a26f024 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:23.866928 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b502c-5d14-4af6-9784-695b1a26f024 None admin] RBAC: Adding query filter params (name=member) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:23.867431 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b502c-5d14-4af6-9784-695b1a26f024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:23.869090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b502c-5d14-4af6-9784-695b1a26f024 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:23.877995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 213/424] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:14:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:23.885074 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b502c-5d14-4af6-9784-695b1a26f024 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:23.972105 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c4b88-ab7c-498a-acf7-f0cbb99929bc None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:24.051110 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b502c-5d14-4af6-9784-695b1a26f024 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:24.060030 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefaa3d9-857f-46dc-bc8e-e04834e10797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:24.060682 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefaa3d9-857f-46dc-bc8e-e04834e10797 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:24.060968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefaa3d9-857f-46dc-bc8e-e04834e10797 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:24.061243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefaa3d9-857f-46dc-bc8e-e04834e10797 None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:24.062413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefaa3d9-857f-46dc-bc8e-e04834e10797 None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:24.064036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefaa3d9-857f-46dc-bc8e-e04834e10797 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:24.064605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:24.066009 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 212/425] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:23 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:24.074429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefaa3d9-857f-46dc-bc8e-e04834e10797 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:24.091247 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d891bef-9a29-4f42-aa89-f06bd317dc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:24.092145 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d891bef-9a29-4f42-aa89-f06bd317dc51 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:24.092643 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d891bef-9a29-4f42-aa89-f06bd317dc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:24.093122 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d891bef-9a29-4f42-aa89-f06bd317dc51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:24.130406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:24.162927 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d891bef-9a29-4f42-aa89-f06bd317dc51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:24.164728 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d891bef-9a29-4f42-aa89-f06bd317dc51 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:24.271410 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 214/426] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:24.273069 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a355a-67c5-4346-9f4b-98d698103d9c None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:24.273361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a355a-67c5-4346-9f4b-98d698103d9c None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:24.273642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a355a-67c5-4346-9f4b-98d698103d9c None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:24.276290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 215/427] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:24.276973 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a355a-67c5-4346-9f4b-98d698103d9c None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:24.285617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:24.287876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefaa3d9-857f-46dc-bc8e-e04834e10797 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:24.324617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfa887-b3ba-4c70-a0ae-9e6636d94950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:24.325635 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfa887-b3ba-4c70-a0ae-9e6636d94950 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:24.326135 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfa887-b3ba-4c70-a0ae-9e6636d94950 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:24.326602 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfa887-b3ba-4c70-a0ae-9e6636d94950 None admin] PATH_INFO: `/v3/groups/bb53a81c848a4c60968050d0c87a771d` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:24.327937 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfa887-b3ba-4c70-a0ae-9e6636d94950 None admin] RBAC: Authorizing `identity:get_group(group_id=bb53a81c848a4c60968050d0c87a771d)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:24.329640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfa887-b3ba-4c70-a0ae-9e6636d94950 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:24.336380 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 213/428] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 5 14:14:24 2022] GET /identity/v3/groups/bb53a81c848a4c60968050d0c87a771d => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:24.338111 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d759c-e236-42d0-97d6-8fe6b14f9db1 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:24.338635 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d759c-e236-42d0-97d6-8fe6b14f9db1 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:24.339432 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d759c-e236-42d0-97d6-8fe6b14f9db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:24.358224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-705d759c-e236-42d0-97d6-8fe6b14f9db1 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:24.359792 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-705d759c-e236-42d0-97d6-8fe6b14f9db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:24.383023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a355a-67c5-4346-9f4b-98d698103d9c None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:24.419390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2a916-98fa-4199-b06d-52991129cc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:24.420122 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2a916-98fa-4199-b06d-52991129cc64 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:24.420478 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2a916-98fa-4199-b06d-52991129cc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:24.420846 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2a916-98fa-4199-b06d-52991129cc64 None admin] PATH_INFO: `/v3/projects/2269c4a7f991405497de468e3a9a745a` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:24.422043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2a916-98fa-4199-b06d-52991129cc64 None admin] RBAC: Authorizing `identity:get_project(project_id=2269c4a7f991405497de468e3a9a745a)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:24.423752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2a916-98fa-4199-b06d-52991129cc64 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:24.425621 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 216/429] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:24 2022] GET /identity/v3/projects/2269c4a7f991405497de468e3a9a745a => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:24.430699 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2a916-98fa-4199-b06d-52991129cc64 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:24.440571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 214/430] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:24.441732 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d759c-e236-42d0-97d6-8fe6b14f9db1 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:24.491272 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:24.519413 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:24.543382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2a916-98fa-4199-b06d-52991129cc64 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:24.587903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d759c-e236-42d0-97d6-8fe6b14f9db1 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:24.623701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969b8bd-f7d5-40b7-ba3b-e0fb0aabc756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:24.624659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969b8bd-f7d5-40b7-ba3b-e0fb0aabc756 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:24.625122 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969b8bd-f7d5-40b7-ba3b-e0fb0aabc756 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:24.625573 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969b8bd-f7d5-40b7-ba3b-e0fb0aabc756 None admin] PATH_INFO: `/v3/groups/bb53a81c848a4c60968050d0c87a771d` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:24.626902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969b8bd-f7d5-40b7-ba3b-e0fb0aabc756 None admin] RBAC: Authorizing `identity:get_group(group_id=bb53a81c848a4c60968050d0c87a771d)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:24.628527 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969b8bd-f7d5-40b7-ba3b-e0fb0aabc756 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:24.630287 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 215/431] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 5 14:14:24 2022] GET /identity/v3/groups/bb53a81c848a4c60968050d0c87a771d => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:24.636176 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969b8bd-f7d5-40b7-ba3b-e0fb0aabc756 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:24.638519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b46924-1d25-48b4-9ab1-c819b4557022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:24.641174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b46924-1d25-48b4-9ab1-c819b4557022 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:24.641642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b46924-1d25-48b4-9ab1-c819b4557022 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:24.642111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b46924-1d25-48b4-9ab1-c819b4557022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:24.643695 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b46924-1d25-48b4-9ab1-c819b4557022 None admin] RBAC: Adding query filter params (group.id=bb53a81c848a4c60968050d0c87a771d, role.id=1182ffc3e9f9477db2003500b7bfd402, scope.project.id=2269c4a7f991405497de468e3a9a745a) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:24.647703 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b46924-1d25-48b4-9ab1-c819b4557022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:24.649390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b46924-1d25-48b4-9ab1-c819b4557022 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:24.666837 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 217/432] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 5 14:14:24 2022] GET /identity/v3/role_assignments?group.id=bb53a81c848a4c60968050d0c87a771d&scope.project.id=2269c4a7f991405497de468e3a9a745a&role.id=1182ffc3e9f9477db2003500b7bfd402 => generated 611 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:24.766761 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:24.773289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969b8bd-f7d5-40b7-ba3b-e0fb0aabc756 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:24.812088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20d423-a98c-4821-9f31-bfd03abfc91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:24.813023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20d423-a98c-4821-9f31-bfd03abfc91a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:24.813506 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20d423-a98c-4821-9f31-bfd03abfc91a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:24.813981 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20d423-a98c-4821-9f31-bfd03abfc91a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:24.815011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f20d423-a98c-4821-9f31-bfd03abfc91a None admin] RBAC: Adding query filter params (group.id=bb53a81c848a4c60968050d0c87a771d, role.id=5812e6538f3f4bdfbfbe11cd25c02957, scope.project.id=2269c4a7f991405497de468e3a9a745a) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:24.815774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f20d423-a98c-4821-9f31-bfd03abfc91a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:24.817450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f20d423-a98c-4821-9f31-bfd03abfc91a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:24.821524 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b46924-1d25-48b4-9ab1-c819b4557022 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:24.825511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 216/433] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 5 14:14:24 2022] GET /identity/v3/role_assignments?group.id=bb53a81c848a4c60968050d0c87a771d&scope.project.id=2269c4a7f991405497de468e3a9a745a&role.id=5812e6538f3f4bdfbfbe11cd25c02957 => generated 611 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:24.877335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7fc44-ea3b-4072-816f-b2ef615d5470 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:24.877981 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7fc44-ea3b-4072-816f-b2ef615d5470 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:24.878561 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7fc44-ea3b-4072-816f-b2ef615d5470 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:24.880386 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 217/434] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:14:24.888003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c56a81-c532-42c6-9ed4-95f63113eda9 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:24.888291 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c56a81-c532-42c6-9ed4-95f63113eda9 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:24.888561 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c56a81-c532-42c6-9ed4-95f63113eda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:24.913125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-55c56a81-c532-42c6-9ed4-95f63113eda9 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:24.914580 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-55c56a81-c532-42c6-9ed4-95f63113eda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:24.982383 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b46924-1d25-48b4-9ab1-c819b4557022 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:25.000871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 218/435] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:14:25.009277 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ada3c-79c7-4c8e-8969-7a42a830ea26 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:25.010004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ada3c-79c7-4c8e-8969-7a42a830ea26 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:25.010511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ada3c-79c7-4c8e-8969-7a42a830ea26 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:25.012266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 219/436] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:25 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:25.018372 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15846c6-19a1-4c27-807b-488985081839 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:25.018955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15846c6-19a1-4c27-807b-488985081839 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:25.019456 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15846c6-19a1-4c27-807b-488985081839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:25.025466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abad6f-a0bd-499a-8639-6c8c85728c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:25.026105 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abad6f-a0bd-499a-8639-6c8c85728c96 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:25.026394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abad6f-a0bd-499a-8639-6c8c85728c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:25.026678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abad6f-a0bd-499a-8639-6c8c85728c96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:25.034173 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:25.041238 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-f15846c6-19a1-4c27-807b-488985081839 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:25.042440 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f15846c6-19a1-4c27-807b-488985081839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:25.042677 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32abad6f-a0bd-499a-8639-6c8c85728c96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:25.047792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-32abad6f-a0bd-499a-8639-6c8c85728c96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:25.047792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:25.047792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:25.047792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:25.047792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:25.047792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:25.047792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:25.047792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:25.047792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:25.047792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:25.047792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:25.047792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:25.047792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:25.047792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:25.047792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:25.047792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:25.048720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:25.048720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:25.048720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:25.048720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:25.048720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:25.048720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:25.048720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:25.048720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:25.048720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:25.048720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:25.048720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:25.048720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:25.048720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:25.048720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:25.048720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:25.048720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:25.049515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:25.049515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:25.049515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:25.049515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:25.049515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:25.049515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:25.049515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:25.049515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:25.049515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:25.049515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:25.049515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 14:14:25.049515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:25.050128 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 218/437] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:25.056530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abad6f-a0bd-499a-8639-6c8c85728c96 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:25.135482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 220/438] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:14:25.181320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abad6f-a0bd-499a-8639-6c8c85728c96 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:25.221281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc32553-ad45-4fa5-ac1d-9d8796ba7db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:25.221541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc32553-ad45-4fa5-ac1d-9d8796ba7db4 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:25.222354 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc32553-ad45-4fa5-ac1d-9d8796ba7db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:25.222354 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc32553-ad45-4fa5-ac1d-9d8796ba7db4 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:25.222871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc32553-ad45-4fa5-ac1d-9d8796ba7db4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:25.223192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc32553-ad45-4fa5-ac1d-9d8796ba7db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:25.224638 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc32553-ad45-4fa5-ac1d-9d8796ba7db4 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:25.234184 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 219/439] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:14:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:25.239925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15846c6-19a1-4c27-807b-488985081839 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:25.250497 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:25.280212 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc32553-ad45-4fa5-ac1d-9d8796ba7db4 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:25.357335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:25.365159 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15846c6-19a1-4c27-807b-488985081839 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:25.409490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc32553-ad45-4fa5-ac1d-9d8796ba7db4 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:25.409799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afda19-b81e-4ec9-bc34-c4057b71caae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:25.410399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afda19-b81e-4ec9-bc34-c4057b71caae None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:25.410676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afda19-b81e-4ec9-bc34-c4057b71caae None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:25.410996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afda19-b81e-4ec9-bc34-c4057b71caae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:25.448921 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575bfa2b-90aa-4bce-ad9b-4bfab90a6fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:25.449565 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bfa2b-90aa-4bce-ad9b-4bfab90a6fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:25.449849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bfa2b-90aa-4bce-ad9b-4bfab90a6fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:25.450126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bfa2b-90aa-4bce-ad9b-4bfab90a6fd2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:25.461493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575bfa2b-90aa-4bce-ad9b-4bfab90a6fd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:25.467112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-575bfa2b-90aa-4bce-ad9b-4bfab90a6fd2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:14:25.467112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:25.467112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:25.467112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:25.467112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:25.467112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:25.467112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:25.467112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:25.467112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:25.467112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:25.467112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:25.467112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:25.467112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:25.467112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:25.467112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:25.467112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:25.468126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:25.468126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:25.468126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:25.468126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:25.468126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:25.468126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:25.468126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:25.468126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:25.468126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:25.468126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:25.468126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:25.468126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:25.468126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:25.468126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:25.468126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:25.468126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:25.468935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:25.468935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:25.468935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:25.468935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:25.468935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:25.468935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:25.468935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:25.468935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:25.468935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:25.468935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:25.468935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:14:25.468935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:25.468935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 220/440] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:14:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:14:25.469615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575bfa2b-90aa-4bce-ad9b-4bfab90a6fd2 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:25.501226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afda19-b81e-4ec9-bc34-c4057b71caae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:25.503004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afda19-b81e-4ec9-bc34-c4057b71caae None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:25.512878 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:25.575511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575bfa2b-90aa-4bce-ad9b-4bfab90a6fd2 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:25.595721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:25.606895 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 221/441] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 14:14:25.606895 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afda19-b81e-4ec9-bc34-c4057b71caae None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:25.612060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6243a78-455f-4873-8023-52e16957af0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:25.612060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6243a78-455f-4873-8023-52e16957af0b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:25.612060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6243a78-455f-4873-8023-52e16957af0b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:25.612060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6243a78-455f-4873-8023-52e16957af0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:25.621154 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6243a78-455f-4873-8023-52e16957af0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:25.626380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-c6243a78-455f-4873-8023-52e16957af0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:14:25.626380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:25.626380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:25.626380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:25.626380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:25.626380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:25.626380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:25.626380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:25.626380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:25.626380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:25.626380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:25.626380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:25.626380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:25.626380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:25.626380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:25.626380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:25.627359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:25.627359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:25.627359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:25.627359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:25.627359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:25.627359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:25.627359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:25.627359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:25.627359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:25.627359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:25.627359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:25.627359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:25.627359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:25.627359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:25.627359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:25.627359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:25.628773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:25.628773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:25.628773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:25.628773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:25.628773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:25.628773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:25.628773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:25.628773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:25.628773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:25.628773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:25.628773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:14:25.628773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:25.628773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 221/442] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:14:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:25.628773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6243a78-455f-4873-8023-52e16957af0b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:25.708082 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afda19-b81e-4ec9-bc34-c4057b71caae None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:25.716624 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:25.727352 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6243a78-455f-4873-8023-52e16957af0b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:25.743854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b928a3-be3e-49c3-b419-9e4790bb36db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:25.745116 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b928a3-be3e-49c3-b419-9e4790bb36db None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:25.745618 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b928a3-be3e-49c3-b419-9e4790bb36db None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:25.745901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b928a3-be3e-49c3-b419-9e4790bb36db None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:25.746560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b928a3-be3e-49c3-b419-9e4790bb36db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:25.746891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b928a3-be3e-49c3-b419-9e4790bb36db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:25.748302 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b928a3-be3e-49c3-b419-9e4790bb36db None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:25.756074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 222/443] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:25.756844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b928a3-be3e-49c3-b419-9e4790bb36db None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:25.762732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590a13b-344b-410e-81ce-a47462de093a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:25.763382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590a13b-344b-410e-81ce-a47462de093a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:25.763661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590a13b-344b-410e-81ce-a47462de093a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:25.763939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590a13b-344b-410e-81ce-a47462de093a None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:25.765049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6590a13b-344b-410e-81ce-a47462de093a None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:25.766490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6590a13b-344b-410e-81ce-a47462de093a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:25.768311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 222/444] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:25 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:25.768916 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590a13b-344b-410e-81ce-a47462de093a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:25.816356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:25.859970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b928a3-be3e-49c3-b419-9e4790bb36db None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:25.865981 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590a13b-344b-410e-81ce-a47462de093a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:25.893893 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4207c-bc4c-4a48-b7c4-968b8df7f774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:25.894543 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4207c-bc4c-4a48-b7c4-968b8df7f774 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:25.894845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4207c-bc4c-4a48-b7c4-968b8df7f774 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:25.895117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4207c-bc4c-4a48-b7c4-968b8df7f774 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:25.895770 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4207c-bc4c-4a48-b7c4-968b8df7f774 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:25.896066 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4207c-bc4c-4a48-b7c4-968b8df7f774 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:25.897433 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4207c-bc4c-4a48-b7c4-968b8df7f774 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:25.899984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd0cce-5a2e-416c-81be-8b9691a9fb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:25.900586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd0cce-5a2e-416c-81be-8b9691a9fb9e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:25.900869 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd0cce-5a2e-416c-81be-8b9691a9fb9e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:25.901141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd0cce-5a2e-416c-81be-8b9691a9fb9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:25.906596 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 223/445] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:25.907427 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4207c-bc4c-4a48-b7c4-968b8df7f774 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:25.938153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:25.971494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd0cce-5a2e-416c-81be-8b9691a9fb9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:25.972972 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd0cce-5a2e-416c-81be-8b9691a9fb9e None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:26.006307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4207c-bc4c-4a48-b7c4-968b8df7f774 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:26.019638 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:26.040063 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cec74-b1b3-46cd-b9f2-82d813764236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:26.041230 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cec74-b1b3-46cd-b9f2-82d813764236 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:26.041513 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cec74-b1b3-46cd-b9f2-82d813764236 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:26.041789 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cec74-b1b3-46cd-b9f2-82d813764236 None admin] PATH_INFO: `/v3/groups/bb53a81c848a4c60968050d0c87a771d` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:26.042942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cec74-b1b3-46cd-b9f2-82d813764236 None admin] RBAC: Authorizing `identity:get_group(group_id=bb53a81c848a4c60968050d0c87a771d)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:26.044376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cec74-b1b3-46cd-b9f2-82d813764236 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:26.046089 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 224/446] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 5 14:14:25 2022] GET /identity/v3/groups/bb53a81c848a4c60968050d0c87a771d => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:26.046676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cec74-b1b3-46cd-b9f2-82d813764236 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:26.070393 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 223/447] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:26.071249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd0cce-5a2e-416c-81be-8b9691a9fb9e None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:26.145327 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:26.148947 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cec74-b1b3-46cd-b9f2-82d813764236 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:26.173513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd0cce-5a2e-416c-81be-8b9691a9fb9e None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:26.182973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaaabff-f80d-4d2a-abec-58e7716b3342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:26.183580 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaaabff-f80d-4d2a-abec-58e7716b3342 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:26.183864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaaabff-f80d-4d2a-abec-58e7716b3342 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:26.184140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaaabff-f80d-4d2a-abec-58e7716b3342 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:26.207926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c812f57-f874-438d-aed7-7abc582b3d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:26.208531 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c812f57-f874-438d-aed7-7abc582b3d8f None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:26.208818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c812f57-f874-438d-aed7-7abc582b3d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:26.209084 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c812f57-f874-438d-aed7-7abc582b3d8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:26.209803 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c812f57-f874-438d-aed7-7abc582b3d8f None admin] RBAC: Adding query filter params (group.id=bb53a81c848a4c60968050d0c87a771d, role.id=5812e6538f3f4bdfbfbe11cd25c02957, scope.project.id=e5e3e090ce8348c19e0ddc4994094302) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:26.210109 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c812f57-f874-438d-aed7-7abc582b3d8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:26.211608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c812f57-f874-438d-aed7-7abc582b3d8f None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:26.218159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 224/448] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 5 14:14:26 2022] GET /identity/v3/role_assignments?group.id=bb53a81c848a4c60968050d0c87a771d&scope.project.id=e5e3e090ce8348c19e0ddc4994094302&role.id=5812e6538f3f4bdfbfbe11cd25c02957 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:26.218964 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c812f57-f874-438d-aed7-7abc582b3d8f None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:26.241870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:26.260177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaaabff-f80d-4d2a-abec-58e7716b3342 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:26.261659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaaabff-f80d-4d2a-abec-58e7716b3342 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:26.346878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c812f57-f874-438d-aed7-7abc582b3d8f None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:26.368127 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:26.374410 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 225/449] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:26.380282 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaaabff-f80d-4d2a-abec-58e7716b3342 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:26.393006 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94261c0-0871-47ba-a9d6-a957da679d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:26.393651 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94261c0-0871-47ba-a9d6-a957da679d5f None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:26.393937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94261c0-0871-47ba-a9d6-a957da679d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:26.394221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94261c0-0871-47ba-a9d6-a957da679d5f None admin] PATH_INFO: `/v3/projects/e5e3e090ce8348c19e0ddc4994094302` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:26.395437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94261c0-0871-47ba-a9d6-a957da679d5f None admin] RBAC: Authorizing `identity:get_project(project_id=e5e3e090ce8348c19e0ddc4994094302)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:26.396833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94261c0-0871-47ba-a9d6-a957da679d5f None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:26.398460 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 225/450] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:26 2022] GET /identity/v3/projects/e5e3e090ce8348c19e0ddc4994094302 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:26.405025 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94261c0-0871-47ba-a9d6-a957da679d5f None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:26.503914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:26.513508 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaaabff-f80d-4d2a-abec-58e7716b3342 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:26.524769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94261c0-0871-47ba-a9d6-a957da679d5f None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:26.556433 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02cd19-73bd-4506-9619-c1c582a65697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:26.557076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02cd19-73bd-4506-9619-c1c582a65697 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:26.557360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02cd19-73bd-4506-9619-c1c582a65697 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:26.557632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02cd19-73bd-4506-9619-c1c582a65697 None admin] PATH_INFO: `/v3/projects/f189779a37444439ac0c4ef7b42cc427` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:26.558760 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02cd19-73bd-4506-9619-c1c582a65697 None admin] RBAC: Authorizing `identity:get_project(project_id=f189779a37444439ac0c4ef7b42cc427)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:26.560318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02cd19-73bd-4506-9619-c1c582a65697 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:26.561951 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 226/451] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:14:26 2022] GET /identity/v3/projects/f189779a37444439ac0c4ef7b42cc427 => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:26.565846 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b19f6f-46ee-4e45-84db-70ca349f9be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:26.566485 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b19f6f-46ee-4e45-84db-70ca349f9be1 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:26.566769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b19f6f-46ee-4e45-84db-70ca349f9be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:26.567096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b19f6f-46ee-4e45-84db-70ca349f9be1 None admin] PATH_INFO: `/v3/groups/bb53a81c848a4c60968050d0c87a771d` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:26.568417 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02cd19-73bd-4506-9619-c1c582a65697 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:26.568554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b19f6f-46ee-4e45-84db-70ca349f9be1 None admin] RBAC: Authorizing `identity:get_group(group_id=bb53a81c848a4c60968050d0c87a771d)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:26.569918 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b19f6f-46ee-4e45-84db-70ca349f9be1 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:26.571663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 226/452] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 5 14:14:26 2022] GET /identity/v3/groups/bb53a81c848a4c60968050d0c87a771d => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:26.577681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b19f6f-46ee-4e45-84db-70ca349f9be1 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:26.634771 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:26.668601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02cd19-73bd-4506-9619-c1c582a65697 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:26.675636 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b19f6f-46ee-4e45-84db-70ca349f9be1 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:26.701599 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc37e7-902c-472c-976f-2a4ea7862c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:26.702240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc37e7-902c-472c-976f-2a4ea7862c43 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:26.702506 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc37e7-902c-472c-976f-2a4ea7862c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:26.702831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc37e7-902c-472c-976f-2a4ea7862c43 None admin] PATH_INFO: `/v3/groups/afab53ef69d74dc7b83e006ec792f927` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:26.704071 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc37e7-902c-472c-976f-2a4ea7862c43 None admin] RBAC: Authorizing `identity:get_group(group_id=afab53ef69d74dc7b83e006ec792f927)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:26.705503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc37e7-902c-472c-976f-2a4ea7862c43 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:26.707080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 227/453] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 5 14:14:26 2022] GET /identity/v3/groups/afab53ef69d74dc7b83e006ec792f927 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:26.709212 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5c537-4ddd-407a-8333-a91b80aca05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:26.709816 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5c537-4ddd-407a-8333-a91b80aca05b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:26.710101 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5c537-4ddd-407a-8333-a91b80aca05b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:26.710378 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5c537-4ddd-407a-8333-a91b80aca05b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:26.711142 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5c537-4ddd-407a-8333-a91b80aca05b None admin] RBAC: Adding query filter params (group.id=bb53a81c848a4c60968050d0c87a771d, role.id=1182ffc3e9f9477db2003500b7bfd402, scope.project.id=e5e3e090ce8348c19e0ddc4994094302) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:26.711451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5c537-4ddd-407a-8333-a91b80aca05b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:26.713453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc37e7-902c-472c-976f-2a4ea7862c43 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:26.713579 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5c537-4ddd-407a-8333-a91b80aca05b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:26.719332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 227/454] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 5 14:14:26 2022] GET /identity/v3/role_assignments?group.id=bb53a81c848a4c60968050d0c87a771d&scope.project.id=e5e3e090ce8348c19e0ddc4994094302&role.id=1182ffc3e9f9477db2003500b7bfd402 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:26.816627 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc37e7-902c-472c-976f-2a4ea7862c43 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:26.850935 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca3057-59b9-4fdb-addb-30a881dd7182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:26.851545 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca3057-59b9-4fdb-addb-30a881dd7182 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:26.852560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca3057-59b9-4fdb-addb-30a881dd7182 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:26.852560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca3057-59b9-4fdb-addb-30a881dd7182 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:26.852801 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca3057-59b9-4fdb-addb-30a881dd7182 None admin] RBAC: Adding query filter params (group.id=afab53ef69d74dc7b83e006ec792f927, role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=f189779a37444439ac0c4ef7b42cc427) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:26.853656 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca3057-59b9-4fdb-addb-30a881dd7182 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:26.855168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca3057-59b9-4fdb-addb-30a881dd7182 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:26.857000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:26.861447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 228/455] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 5 14:14:26 2022] GET /identity/v3/role_assignments?group.id=afab53ef69d74dc7b83e006ec792f927&scope.project.id=f189779a37444439ac0c4ef7b42cc427&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:29.146002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c2b2c-bf53-4969-995d-f0a03dbd1e89 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:29.146507 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c2b2c-bf53-4969-995d-f0a03dbd1e89 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:29.146606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c2b2c-bf53-4969-995d-f0a03dbd1e89 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:29.148078 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 228/456] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:29.154024 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93c053-f48d-423e-b2b9-ce9a4da1adc9 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:29.154884 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93c053-f48d-423e-b2b9-ce9a4da1adc9 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:29.155364 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93c053-f48d-423e-b2b9-ce9a4da1adc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:29.179756 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-ba93c053-f48d-423e-b2b9-ce9a4da1adc9 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:29.181321 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ba93c053-f48d-423e-b2b9-ce9a4da1adc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:29.282064 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 229/457] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:29.330734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94425da5-3981-4ee3-a9d5-9d78c216b594 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:29.331380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94425da5-3981-4ee3-a9d5-9d78c216b594 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:29.331864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94425da5-3981-4ee3-a9d5-9d78c216b594 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:29.333714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 229/458] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:29.339104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc5845-af11-46a7-a36c-1ac72b05d1ab None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:29.339614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc5845-af11-46a7-a36c-1ac72b05d1ab None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:29.340129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc5845-af11-46a7-a36c-1ac72b05d1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:29.367884 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-96cc5845-af11-46a7-a36c-1ac72b05d1ab None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:29.369445 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-96cc5845-af11-46a7-a36c-1ac72b05d1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:29.452731 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 230/459] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:29.618330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94425da5-3981-4ee3-a9d5-9d78c216b594 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:29.635024 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:29.749574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94425da5-3981-4ee3-a9d5-9d78c216b594 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:29.755160 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0ed2e-dc93-435e-b0bc-cb6e10ea2c3c None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:29.755447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0ed2e-dc93-435e-b0bc-cb6e10ea2c3c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:29.755725 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0ed2e-dc93-435e-b0bc-cb6e10ea2c3c None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:29.757312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 231/460] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:29.763649 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7a644-2d02-481a-a8bc-04505a4632c6 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:29.763934 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7a644-2d02-481a-a8bc-04505a4632c6 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:29.764209 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7a644-2d02-481a-a8bc-04505a4632c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:29.794320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-33c7a644-2d02-481a-a8bc-04505a4632c6 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:29.797241 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-33c7a644-2d02-481a-a8bc-04505a4632c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:29.818582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7288439-18e5-4a6a-b172-2a1b8c2cf47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:29.819280 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7288439-18e5-4a6a-b172-2a1b8c2cf47c None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:29.819562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7288439-18e5-4a6a-b172-2a1b8c2cf47c None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:29.819848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7288439-18e5-4a6a-b172-2a1b8c2cf47c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:29.831504 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7288439-18e5-4a6a-b172-2a1b8c2cf47c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:29.832971 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7288439-18e5-4a6a-b172-2a1b8c2cf47c None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:29.847454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-f7288439-18e5-4a6a-b172-2a1b8c2cf47c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:29.847454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:29.847454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:29.847454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:29.847454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:29.847454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:29.847454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:29.847454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:29.847454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:29.847454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:29.847454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:29.847454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:29.847454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:29.847454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:29.847454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:29.847454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:29.848382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:29.848382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:29.848382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:29.848382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:29.848382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:29.848382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:29.848382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:29.848382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:29.848382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:29.848382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:29.848382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:29.848382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:29.848382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:29.848382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:29.848382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:29.848382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:29.848382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:29.854570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:29.854570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:29.854570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:29.854570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 230/461] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:29.854570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7288439-18e5-4a6a-b172-2a1b8c2cf47c None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:29.878113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:29.897711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 232/462] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:29.899374 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7a644-2d02-481a-a8bc-04505a4632c6 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:29.931034 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:29.973161 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7288439-18e5-4a6a-b172-2a1b8c2cf47c None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:30.009525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31143e-42ae-453d-8a77-cdf3ce7f8717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:30.010162 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31143e-42ae-453d-8a77-cdf3ce7f8717 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:30.010449 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31143e-42ae-453d-8a77-cdf3ce7f8717 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:30.010728 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31143e-42ae-453d-8a77-cdf3ce7f8717 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:30.022150 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31143e-42ae-453d-8a77-cdf3ce7f8717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:30.023755 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31143e-42ae-453d-8a77-cdf3ce7f8717 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:30.026845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7a644-2d02-481a-a8bc-04505a4632c6 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:30.041061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-7e31143e-42ae-453d-8a77-cdf3ce7f8717 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:30.041061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:30.041061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:30.041061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:30.041061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:30.041061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:30.041061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:30.041061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:30.041061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:30.041061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:30.041061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:30.041061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:30.041061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:30.041061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:30.041061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:30.041061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:30.042125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:30.042125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:30.042125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:30.042125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:30.042125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:30.042125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:30.042125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:30.042125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:30.042125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:30.042125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:30.042125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:30.042125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:30.042125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:30.042125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:30.042125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:30.042125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:30.042125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:30.043003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:30.043003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:30.043003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:30.047114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 231/463] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:30.048444 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863d540-378b-492e-8252-4e0cc0e922e8 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:30.048626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863d540-378b-492e-8252-4e0cc0e922e8 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:30.048870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863d540-378b-492e-8252-4e0cc0e922e8 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:30.050460 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 232/464] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:30.051216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863d540-378b-492e-8252-4e0cc0e922e8 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:30.077394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5a308-d49e-4ec5-8172-ff6688acd778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:30.078061 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5a308-d49e-4ec5-8172-ff6688acd778 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:30.078353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5a308-d49e-4ec5-8172-ff6688acd778 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:30.078627 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5a308-d49e-4ec5-8172-ff6688acd778 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:30.079892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5a308-d49e-4ec5-8172-ff6688acd778 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:30.080199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5a308-d49e-4ec5-8172-ff6688acd778 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:30.081609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5a308-d49e-4ec5-8172-ff6688acd778 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:30.100520 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 233/465] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:30.100520 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad89b2-5918-4522-a58d-01de1a426659 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:30.100520 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad89b2-5918-4522-a58d-01de1a426659 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:30.100520 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad89b2-5918-4522-a58d-01de1a426659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:30.117398 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-f5ad89b2-5918-4522-a58d-01de1a426659 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:30.119066 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f5ad89b2-5918-4522-a58d-01de1a426659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:30.167340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:30.186490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 234/466] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:30.187329 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad89b2-5918-4522-a58d-01de1a426659 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:30.203518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863d540-378b-492e-8252-4e0cc0e922e8 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:30.227592 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:30.252076 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777d738-fc7b-4f20-a107-18c1f09348b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:30.253187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777d738-fc7b-4f20-a107-18c1f09348b6 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:30.253618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777d738-fc7b-4f20-a107-18c1f09348b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:30.254356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777d738-fc7b-4f20-a107-18c1f09348b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:30.256398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777d738-fc7b-4f20-a107-18c1f09348b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:30.256860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777d738-fc7b-4f20-a107-18c1f09348b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:30.258388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777d738-fc7b-4f20-a107-18c1f09348b6 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:30.276641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 233/467] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:30.278106 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777d738-fc7b-4f20-a107-18c1f09348b6 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:30.301673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad89b2-5918-4522-a58d-01de1a426659 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:30.348636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3f8a2-9987-4179-b0a2-c6b2f6eacfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:30.349722 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3f8a2-9987-4179-b0a2-c6b2f6eacfce None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:30.350145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3f8a2-9987-4179-b0a2-c6b2f6eacfce None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:30.351172 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3f8a2-9987-4179-b0a2-c6b2f6eacfce None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:30.351956 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3f8a2-9987-4179-b0a2-c6b2f6eacfce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:30.354530 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3f8a2-9987-4179-b0a2-c6b2f6eacfce None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:30.387568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777d738-fc7b-4f20-a107-18c1f09348b6 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:30.422539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc5cdd-b0af-4e12-8fd8-cbb1702922ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:30.423229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc5cdd-b0af-4e12-8fd8-cbb1702922ec None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:30.423577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc5cdd-b0af-4e12-8fd8-cbb1702922ec None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:30.423917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc5cdd-b0af-4e12-8fd8-cbb1702922ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:30.434973 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc5cdd-b0af-4e12-8fd8-cbb1702922ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:30.436422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc5cdd-b0af-4e12-8fd8-cbb1702922ec None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:30.451380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-1ccc5cdd-b0af-4e12-8fd8-cbb1702922ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:30.451380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:30.451380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:30.451380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:30.451380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:30.451380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:30.451380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:30.451380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:30.451380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:30.451380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:30.451380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:30.451380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:30.451380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:30.451380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:30.451380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:30.451380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:30.452309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:30.452309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:30.452309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:30.452309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:30.452309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:30.452309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:30.452309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:30.452309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:30.452309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:30.452309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:30.452309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:30.452309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:30.452309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:30.452309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:30.452309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:30.452309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:30.452309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:30.453158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:30.453158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:30.453158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:30.454919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 234/468] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:30.455924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc5cdd-b0af-4e12-8fd8-cbb1702922ec None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:30.470929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 235/469] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:14:30 2022] POST /identity/v3/users => generated 246 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:14:30.470929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3f8a2-9987-4179-b0a2-c6b2f6eacfce None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:30.491778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:30.522333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:30.602347 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc5cdd-b0af-4e12-8fd8-cbb1702922ec None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:30.612593 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3f8a2-9987-4179-b0a2-c6b2f6eacfce None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:30.652746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e31ddb-6404-4886-ad7b-d56b51396a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:30.653364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e31ddb-6404-4886-ad7b-d56b51396a64 None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:30.653643 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e31ddb-6404-4886-ad7b-d56b51396a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:30.653919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e31ddb-6404-4886-ad7b-d56b51396a64 None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:30.654569 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e31ddb-6404-4886-ad7b-d56b51396a64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:30.656084 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e31ddb-6404-4886-ad7b-d56b51396a64 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:30.658326 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee5dea-76b5-4701-a19c-5600339ac8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:30.659235 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee5dea-76b5-4701-a19c-5600339ac8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:30.659513 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee5dea-76b5-4701-a19c-5600339ac8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:30.659780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee5dea-76b5-4701-a19c-5600339ac8f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:30.670165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee5dea-76b5-4701-a19c-5600339ac8f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:30.671672 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee5dea-76b5-4701-a19c-5600339ac8f1 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:30.688644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-f2ee5dea-76b5-4701-a19c-5600339ac8f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:30.688644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:30.688644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:30.688644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:30.688644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:30.688644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:30.688644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:30.688644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:30.688644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:30.688644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:30.688644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:30.688644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:30.688644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:30.688644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:30.688644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:30.688644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:30.689723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:30.689723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:30.689723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:30.689723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:30.689723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:30.689723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:30.689723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:30.689723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:30.689723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:30.689723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:30.689723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:30.689723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:30.689723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:30.689723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:30.689723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:30.689723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:30.689723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:30.690923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:30.690923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:30.690923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:30.690923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 236/470] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:30.690923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee5dea-76b5-4701-a19c-5600339ac8f1 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:30.738544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 235/471] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:14:30 2022] POST /identity/v3/users => generated 248 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:14:30.739560 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e31ddb-6404-4886-ad7b-d56b51396a64 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:30.805247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:30.828567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee5dea-76b5-4701-a19c-5600339ac8f1 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:30.839154 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:30.871606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e31ddb-6404-4886-ad7b-d56b51396a64 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:30.907275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d033f-bfb5-4963-8a35-76716432e9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:30.907908 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d033f-bfb5-4963-8a35-76716432e9b1 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:30.909346 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d033f-bfb5-4963-8a35-76716432e9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:30.909634 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d033f-bfb5-4963-8a35-76716432e9b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:30.910300 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d033f-bfb5-4963-8a35-76716432e9b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:30.910604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d033f-bfb5-4963-8a35-76716432e9b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:30.912061 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d033f-bfb5-4963-8a35-76716432e9b1 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:30.940479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 237/472] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:30.946283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d033f-bfb5-4963-8a35-76716432e9b1 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:30.948563 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd362c-bf30-4bfa-a447-5935c0e99f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:30.949561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd362c-bf30-4bfa-a447-5935c0e99f1f None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:30.949926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd362c-bf30-4bfa-a447-5935c0e99f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:30.951195 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd362c-bf30-4bfa-a447-5935c0e99f1f None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:30.951872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd362c-bf30-4bfa-a447-5935c0e99f1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:30.952185 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd362c-bf30-4bfa-a447-5935c0e99f1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:30.953599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd362c-bf30-4bfa-a447-5935c0e99f1f None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:30.972942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 236/473] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:30.974605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a03ea-2ec4-4ad7-937c-cb17eb41938b None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:30.975016 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a03ea-2ec4-4ad7-937c-cb17eb41938b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:30.975383 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a03ea-2ec4-4ad7-937c-cb17eb41938b None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:30.976790 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 237/474] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:30.979762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a03ea-2ec4-4ad7-937c-cb17eb41938b None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:31.081638 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d033f-bfb5-4963-8a35-76716432e9b1 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:31.112396 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a03ea-2ec4-4ad7-937c-cb17eb41938b None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:31.127504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afedddd-223a-4412-b7fe-3d8a22938628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:31.129241 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afedddd-223a-4412-b7fe-3d8a22938628 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:31.130589 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afedddd-223a-4412-b7fe-3d8a22938628 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:31.131154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afedddd-223a-4412-b7fe-3d8a22938628 None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:31.132074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afedddd-223a-4412-b7fe-3d8a22938628 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:31.133727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afedddd-223a-4412-b7fe-3d8a22938628 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:31.152748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bcf89-0596-4f26-a973-92ebbc270ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:31.153360 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bcf89-0596-4f26-a973-92ebbc270ca3 None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:31.153642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bcf89-0596-4f26-a973-92ebbc270ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:31.153913 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bcf89-0596-4f26-a973-92ebbc270ca3 None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:31.154559 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bcf89-0596-4f26-a973-92ebbc270ca3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:31.156028 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bcf89-0596-4f26-a973-92ebbc270ca3 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:31.290256 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 238/475] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:14:30 2022] POST /identity/v3/users => generated 248 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:14:31.290256 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09367183-358f-407d-8d10-65b224c633e7 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:31.290256 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09367183-358f-407d-8d10-65b224c633e7 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:31.290256 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09367183-358f-407d-8d10-65b224c633e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:31.297640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:31.312120 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:31.313885 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-09367183-358f-407d-8d10-65b224c633e7 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:31.315752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-09367183-358f-407d-8d10-65b224c633e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:31.324893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 238/476] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:14:30 2022] POST /identity/v3/users => generated 249 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:14:31.402174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 239/477] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:31.673905 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bcf89-0596-4f26-a973-92ebbc270ca3 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:31.793607 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bcf89-0596-4f26-a973-92ebbc270ca3 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:31.840874 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298ca89-66d9-44e8-a1d1-ea37cd659f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:31.841555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298ca89-66d9-44e8-a1d1-ea37cd659f22 None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:31.841845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298ca89-66d9-44e8-a1d1-ea37cd659f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:31.842120 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298ca89-66d9-44e8-a1d1-ea37cd659f22 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:31.842831 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298ca89-66d9-44e8-a1d1-ea37cd659f22 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:31.844216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298ca89-66d9-44e8-a1d1-ea37cd659f22 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:31.878896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.sql.core [None req-c298ca89-66d9-44e8-a1d1-ea37cd659f22 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 05 14:14:31.878896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 05 14:14:31.878896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [parameters: {'id': '894f9b83210247779f6842bf2264c083', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 05 14:14:31.878896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81595) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 05 14:14:31.897311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-c298ca89-66d9-44e8-a1d1-ea37cd659f22 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 05 14:14:31.897311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:31.897311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 14:14:31.897311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 14:14:31.897311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 14:14:31.897311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 14:14:31.897311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 14:14:31.897311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 14:14:31.897311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 14:14:31.897311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application conn.query(q) Sep 05 14:14:31.897311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 14:14:31.897311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 14:14:31.897311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 14:14:31.897311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application result.read() Sep 05 14:14:31.897311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return ref.to_dict() Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 14:14:31.898223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application next(self.gen) Sep 05 14:14:31.899203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 05 14:14:31.899203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application yield resource Sep 05 14:14:31.899203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 14:14:31.899203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application next(self.gen) Sep 05 14:14:31.899203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 05 14:14:31.899203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self.session.rollback() Sep 05 14:14:31.899203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 05 14:14:31.899203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self.force_reraise() Sep 05 14:14:31.899203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 05 14:14:31.899203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise self.value Sep 05 14:14:31.899203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 05 14:14:31.899203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 05 14:14:31.899203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 05 14:14:31.899203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application session.commit() Sep 05 14:14:31.899203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 05 14:14:31.899203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 05 14:14:31.899203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 05 14:14:31.900176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._prepare_impl() Sep 05 14:14:31.900176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 05 14:14:31.900176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self.session.flush() Sep 05 14:14:31.900176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 05 14:14:31.900176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._flush(objects) Sep 05 14:14:31.900176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 05 14:14:31.900176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 05 14:14:31.900176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 05 14:14:31.900176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application compat.raise_( Sep 05 14:14:31.900176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 14:14:31.900176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception Sep 05 14:14:31.900176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 05 14:14:31.900176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application flush_context.execute() Sep 05 14:14:31.900176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 05 14:14:31.900176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rec.execute(self) Sep 05 14:14:31.900176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 05 14:14:31.900176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 05 14:14:31.901864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 05 14:14:31.901864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 05 14:14:31.901864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 05 14:14:31.901864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 05 14:14:31.901864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 05 14:14:31.901864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 05 14:14:31.901864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 05 14:14:31.901864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 05 14:14:31.901864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 05 14:14:31.901864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 05 14:14:31.901864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 05 14:14:31.901864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 05 14:14:31.901864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 05 14:14:31.901864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 05 14:14:31.901864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 14:14:31.901864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception Sep 05 14:14:31.902714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 14:14:31.902714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 14:14:31.902714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 14:14:31.902714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 14:14:31.902714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 14:14:31.902714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 14:14:31.902714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 14:14:31.902714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application conn.query(q) Sep 05 14:14:31.902714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 14:14:31.902714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 14:14:31.902714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 14:14:31.902714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application result.read() Sep 05 14:14:31.902714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 14:14:31.902714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 14:14:31.902714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 14:14:31.902714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 14:14:31.904334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 14:14:31.904334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 14:14:31.904334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 14:14:31.904334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 14:14:31.904334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 05 14:14:31.904334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 05 14:14:31.904334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [parameters: {'id': '894f9b83210247779f6842bf2264c083', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 05 14:14:31.904334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 05 14:14:31.904334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:31.904334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:14:31.904334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:31.904334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:31.904334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:31.904334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:31.904334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:31.904334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:31.911159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:31.911159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:31.911159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:31.911159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:31.911159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:31.911159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:31.911159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 05 14:14:31.911159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 05 14:14:31.911159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:31.911159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:31.911159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 05 14:14:31.911159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 05 14:14:31.911159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 05 14:14:31.911159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 05 14:14:31.911159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 05 14:14:31.911159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:31.913156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 239/478] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Sep 5 14:14:31 2022] POST /identity/v3/roles => generated 144 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:31.913293 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09367183-358f-407d-8d10-65b224c633e7 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:32.054919 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09367183-358f-407d-8d10-65b224c633e7 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:32.112406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:32.120348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e2cba-5577-4b6c-9e20-4fbb3078dee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:32.121004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e2cba-5577-4b6c-9e20-4fbb3078dee6 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:32.121287 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e2cba-5577-4b6c-9e20-4fbb3078dee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:32.121566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e2cba-5577-4b6c-9e20-4fbb3078dee6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:32.138687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e2cba-5577-4b6c-9e20-4fbb3078dee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:32.146577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-604e2cba-5577-4b6c-9e20-4fbb3078dee6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:14:32.146577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:32.146577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:32.146577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:32.146577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:32.146577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:32.146577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:32.146577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:32.146577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:32.146577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:32.146577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:32.146577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:32.146577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:32.146577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:32.146577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:32.146577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:32.147471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:32.147471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:32.147471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:32.147471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:32.147471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:32.147471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:32.147471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:32.147471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:32.147471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:32.147471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:32.147471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:32.147471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:32.147471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:32.147471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:32.147471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:32.147471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:32.148842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:32.148842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:32.148842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:32.148842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:32.148842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:32.148842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:32.148842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:32.148842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:32.148842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:32.148842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:32.148842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 14:14:32.148842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:32.148842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 240/479] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:14:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:32.154082 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298ca89-66d9-44e8-a1d1-ea37cd659f22 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:32.244344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:32.290503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298ca89-66d9-44e8-a1d1-ea37cd659f22 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:32.324915 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2080dc-be00-4f27-ad3b-b68aed0f6266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:32.325513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2080dc-be00-4f27-ad3b-b68aed0f6266 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:32.325767 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2080dc-be00-4f27-ad3b-b68aed0f6266 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:32.326041 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2080dc-be00-4f27-ad3b-b68aed0f6266 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:32.326759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2080dc-be00-4f27-ad3b-b68aed0f6266 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:32.327086 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2080dc-be00-4f27-ad3b-b68aed0f6266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:32.328456 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2080dc-be00-4f27-ad3b-b68aed0f6266 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:32.336545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 240/480] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:32.823614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7143f10-a798-49c1-889e-f38952f9662e None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:32.824141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7143f10-a798-49c1-889e-f38952f9662e None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:32.824141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7143f10-a798-49c1-889e-f38952f9662e None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:32.826346 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 241/481] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:32.831335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c1cf4-298a-494b-8b03-a251b0474e9e None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:32.831626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c1cf4-298a-494b-8b03-a251b0474e9e None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:32.831905 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c1cf4-298a-494b-8b03-a251b0474e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:32.863970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-e63c1cf4-298a-494b-8b03-a251b0474e9e None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:32.865200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e63c1cf4-298a-494b-8b03-a251b0474e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:32.957359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 241/482] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:14:33.039623 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e291ae-831e-4d86-9c54-ab6720ef7c2b None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:33.039908 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e291ae-831e-4d86-9c54-ab6720ef7c2b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:33.040166 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e291ae-831e-4d86-9c54-ab6720ef7c2b None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:33.041766 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 242/483] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:33.047137 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc8962-e913-4f7c-bf02-376779d6c9e1 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:33.047427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc8962-e913-4f7c-bf02-376779d6c9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:33.047705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc8962-e913-4f7c-bf02-376779d6c9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:33.071851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-28bc8962-e913-4f7c-bf02-376779d6c9e1 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:33.075637 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-28bc8962-e913-4f7c-bf02-376779d6c9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:33.124355 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:33.140836 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 242/484] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:33.298377 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e291ae-831e-4d86-9c54-ab6720ef7c2b None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:33.424550 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e291ae-831e-4d86-9c54-ab6720ef7c2b None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:33.440963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc8962-e913-4f7c-bf02-376779d6c9e1 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:33.485159 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228395c-fa56-43c4-9848-c12efeecf648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:33.485801 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228395c-fa56-43c4-9848-c12efeecf648 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:33.486081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228395c-fa56-43c4-9848-c12efeecf648 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:33.486355 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228395c-fa56-43c4-9848-c12efeecf648 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:33.508701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228395c-fa56-43c4-9848-c12efeecf648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:33.515448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-2228395c-fa56-43c4-9848-c12efeecf648 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:33.515448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:33.515448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:33.515448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:33.515448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:33.515448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:33.515448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:33.515448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:33.515448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:33.515448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:33.515448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:33.515448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:33.515448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:33.515448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:33.515448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:33.515448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:33.516348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:33.516348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:33.516348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:33.516348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:33.516348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:33.516348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:33.516348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:33.516348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:33.516348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:33.516348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:33.516348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:33.516348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:33.516348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:33.516348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:33.516348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:33.516348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:33.517150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:33.517150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:33.517150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:33.517150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:33.517150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:33.517150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:33.517150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:33.517150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:33.517150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:33.517150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:33.517150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:33.517150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:33.517772 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 243/485] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:33.523911 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228395c-fa56-43c4-9848-c12efeecf648 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:33.612673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:33.616467 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc8962-e913-4f7c-bf02-376779d6c9e1 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:33.678883 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd734a-192f-4265-bc96-d99762d08589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:33.679503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd734a-192f-4265-bc96-d99762d08589 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:33.679787 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd734a-192f-4265-bc96-d99762d08589 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:33.680056 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd734a-192f-4265-bc96-d99762d08589 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:33.697269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228395c-fa56-43c4-9848-c12efeecf648 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:33.701154 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd734a-192f-4265-bc96-d99762d08589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:33.705926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-dedd734a-192f-4265-bc96-d99762d08589 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:33.705926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:33.705926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:33.705926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:33.705926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:33.705926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:33.705926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:33.705926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:33.705926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:33.705926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:33.705926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:33.705926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:33.705926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:33.705926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:33.705926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:33.705926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:33.706805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:33.706805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:33.706805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:33.706805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:33.706805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:33.706805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:33.706805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:33.706805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:33.706805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:33.706805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:33.706805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:33.706805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:33.706805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:33.706805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:33.706805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:33.706805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:33.707615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:33.707615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:33.707615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:33.707615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:33.707615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:33.707615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:33.707615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:33.707615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:33.707615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:33.707615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:33.707615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:33.707615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:33.711828 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 243/486] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:33.715131 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd734a-192f-4265-bc96-d99762d08589 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:33.760229 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fce388-6cd5-4f37-8248-4bc0d6fec866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:33.760838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fce388-6cd5-4f37-8248-4bc0d6fec866 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:33.761122 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fce388-6cd5-4f37-8248-4bc0d6fec866 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:33.761396 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fce388-6cd5-4f37-8248-4bc0d6fec866 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:33.762073 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fce388-6cd5-4f37-8248-4bc0d6fec866 None admin] RBAC: Adding query filter params (name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:33.762377 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fce388-6cd5-4f37-8248-4bc0d6fec866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:33.763790 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fce388-6cd5-4f37-8248-4bc0d6fec866 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:33.788169 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 244/487] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:14:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:33.789809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fce388-6cd5-4f37-8248-4bc0d6fec866 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:33.806810 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:33.874686 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd734a-192f-4265-bc96-d99762d08589 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:33.911006 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a6ab5-e372-433e-9546-c456d2bf0d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:33.911625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a6ab5-e372-433e-9546-c456d2bf0d77 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:33.911962 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a6ab5-e372-433e-9546-c456d2bf0d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:33.912205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a6ab5-e372-433e-9546-c456d2bf0d77 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:33.912872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a6ab5-e372-433e-9546-c456d2bf0d77 None admin] RBAC: Adding query filter params (name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:33.913170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a6ab5-e372-433e-9546-c456d2bf0d77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:33.914529 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a6ab5-e372-433e-9546-c456d2bf0d77 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:33.921286 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fce388-6cd5-4f37-8248-4bc0d6fec866 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:33.924135 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 244/488] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:14:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:33.925516 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efed26a-2783-4a17-b1de-f6b1cf0cc000 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:33.925805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efed26a-2783-4a17-b1de-f6b1cf0cc000 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:33.926078 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efed26a-2783-4a17-b1de-f6b1cf0cc000 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:33.930920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 245/489] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:33.930920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c38f9-acd9-46c4-b34b-4517dd395d99 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:33.930920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c38f9-acd9-46c4-b34b-4517dd395d99 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:33.930920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c38f9-acd9-46c4-b34b-4517dd395d99 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:33.932459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 246/490] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:33.934981 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c38f9-acd9-46c4-b34b-4517dd395d99 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:33.945956 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:33.967639 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e658e21-9fa3-4776-9192-a0cfb19e7f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:33.968346 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e658e21-9fa3-4776-9192-a0cfb19e7f71 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:33.968625 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e658e21-9fa3-4776-9192-a0cfb19e7f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:33.969138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e658e21-9fa3-4776-9192-a0cfb19e7f71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:33.981644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e658e21-9fa3-4776-9192-a0cfb19e7f71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:33.983188 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e658e21-9fa3-4776-9192-a0cfb19e7f71 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:33.998366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-1e658e21-9fa3-4776-9192-a0cfb19e7f71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:33.998366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:33.998366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:33.998366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:33.998366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:33.998366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:33.998366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:33.998366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:33.998366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:33.998366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:33.998366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:33.998366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:33.998366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:33.998366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:33.998366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:33.998366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:33.999342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:33.999342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:33.999342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:33.999342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:33.999342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:33.999342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:33.999342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:33.999342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:33.999342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:33.999342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:33.999342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:33.999342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:33.999342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:33.999342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:33.999342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:33.999342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:33.999342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:34.000199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:34.000199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:34.000199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:34.000199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 245/491] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:34.000942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0542ec9-5466-48c5-9eb6-cc7b1d5c9da1 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:34.000942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0542ec9-5466-48c5-9eb6-cc7b1d5c9da1 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:34.001182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0542ec9-5466-48c5-9eb6-cc7b1d5c9da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:34.019941 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-e0542ec9-5466-48c5-9eb6-cc7b1d5c9da1 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:34.021136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e0542ec9-5466-48c5-9eb6-cc7b1d5c9da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:34.050717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c38f9-acd9-46c4-b34b-4517dd395d99 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:34.060074 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:34.087931 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec938e1-cfed-4ced-b57a-0398c266ea68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:34.088585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec938e1-cfed-4ced-b57a-0398c266ea68 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:34.089020 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec938e1-cfed-4ced-b57a-0398c266ea68 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:34.089882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec938e1-cfed-4ced-b57a-0398c266ea68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:34.095190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 246/492] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:34.096505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b5acd-4c13-4024-bc4f-4f5b2f655108 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:34.096859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b5acd-4c13-4024-bc4f-4f5b2f655108 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:34.097221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b5acd-4c13-4024-bc4f-4f5b2f655108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:34.113663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec938e1-cfed-4ced-b57a-0398c266ea68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:34.115465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec938e1-cfed-4ced-b57a-0398c266ea68 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:34.115811 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-2e6b5acd-4c13-4024-bc4f-4f5b2f655108 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:34.117046 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-2e6b5acd-4c13-4024-bc4f-4f5b2f655108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:34.135813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-6ec938e1-cfed-4ced-b57a-0398c266ea68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:34.135813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:34.135813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:34.135813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:34.135813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:34.135813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:34.135813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:34.135813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:34.135813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:34.135813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:34.135813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:34.135813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:34.135813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:34.135813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:34.135813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:34.135813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:34.136706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:34.136706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:34.136706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:34.136706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:34.136706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:34.136706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:34.136706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:34.136706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:34.136706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:34.136706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:34.136706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:34.136706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:34.136706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:34.136706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:34.136706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:34.136706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:34.136706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:34.137635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:34.137635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:34.137635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:34.140347 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 247/493] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:34.141343 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec938e1-cfed-4ced-b57a-0398c266ea68 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:34.199352 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 247/494] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:34.200148 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b5acd-4c13-4024-bc4f-4f5b2f655108 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:34.246307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:34.267946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec938e1-cfed-4ced-b57a-0398c266ea68 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:34.304701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f2327-6b7b-4354-88c6-d8a270f0af64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:34.305596 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f2327-6b7b-4354-88c6-d8a270f0af64 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:34.306292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f2327-6b7b-4354-88c6-d8a270f0af64 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:34.306747 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f2327-6b7b-4354-88c6-d8a270f0af64 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:34.307659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f2327-6b7b-4354-88c6-d8a270f0af64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:34.308126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f2327-6b7b-4354-88c6-d8a270f0af64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:34.309648 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f2327-6b7b-4354-88c6-d8a270f0af64 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:34.323819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 248/495] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:34.332278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f2327-6b7b-4354-88c6-d8a270f0af64 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:34.374259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:34.375804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b5acd-4c13-4024-bc4f-4f5b2f655108 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:34.418383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17686c2d-d47e-4a29-be04-58299d28fac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:34.419524 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17686c2d-d47e-4a29-be04-58299d28fac4 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:34.420022 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17686c2d-d47e-4a29-be04-58299d28fac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:34.420481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17686c2d-d47e-4a29-be04-58299d28fac4 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:34.421305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17686c2d-d47e-4a29-be04-58299d28fac4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:34.421797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17686c2d-d47e-4a29-be04-58299d28fac4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:34.423537 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17686c2d-d47e-4a29-be04-58299d28fac4 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:34.436974 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 248/496] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:34.438136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17686c2d-d47e-4a29-be04-58299d28fac4 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:34.458899 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f2327-6b7b-4354-88c6-d8a270f0af64 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:34.495728 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b1582-eb4a-434b-a392-c1622d774dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:34.496415 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b1582-eb4a-434b-a392-c1622d774dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:34.496697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b1582-eb4a-434b-a392-c1622d774dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:34.496971 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b1582-eb4a-434b-a392-c1622d774dd2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:34.504418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b1582-eb4a-434b-a392-c1622d774dd2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:34.505873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b1582-eb4a-434b-a392-c1622d774dd2 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:34.519547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-ca7b1582-eb4a-434b-a392-c1622d774dd2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:14:34.519547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:34.519547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:34.519547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:34.519547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:34.519547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:34.519547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:34.519547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:34.519547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:34.519547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:34.519547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:34.519547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:34.519547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:34.519547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:34.519547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:34.519547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:34.520508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:34.521809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:34.521809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:34.521809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:34.521809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:34.521809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:34.521809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:34.521809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:34.521809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:34.521809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:14:34.521809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:34.521809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 249/497] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 5 14:14:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:34.521809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b1582-eb4a-434b-a392-c1622d774dd2 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:34.542467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:34.559054 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17686c2d-d47e-4a29-be04-58299d28fac4 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:34.593628 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a25f3-214a-495a-af20-18df0bf10807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:34.594255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a25f3-214a-495a-af20-18df0bf10807 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:34.594532 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a25f3-214a-495a-af20-18df0bf10807 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:34.594820 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a25f3-214a-495a-af20-18df0bf10807 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:34.604989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a25f3-214a-495a-af20-18df0bf10807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:34.609797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-0f0a25f3-214a-495a-af20-18df0bf10807 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:34.609797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:34.609797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:34.609797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:34.609797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:34.609797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:34.609797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:34.609797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:34.609797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:34.609797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:34.609797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:34.609797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:34.609797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:34.609797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:34.609797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:34.609797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:34.610677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:34.610677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:34.610677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:34.610677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:34.610677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:34.610677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:34.610677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:34.610677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:34.610677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:34.610677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:34.610677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:34.610677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:34.610677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:34.610677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:34.610677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:34.610677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:34.611511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:34.611511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:34.611511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:34.611511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:34.611511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:34.611511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:34.611511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:34.611511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:34.611511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:34.611511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:34.611511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:34.611511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:34.611511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 249/498] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:34.612659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:34.616192 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a25f3-214a-495a-af20-18df0bf10807 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:34.634209 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b1582-eb4a-434b-a392-c1622d774dd2 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:34.673477 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3870bc-5a7c-49ae-a823-2941620f23e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:34.674615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3870bc-5a7c-49ae-a823-2941620f23e4 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:34.675367 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3870bc-5a7c-49ae-a823-2941620f23e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:34.676054 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3870bc-5a7c-49ae-a823-2941620f23e4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:34.686715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3870bc-5a7c-49ae-a823-2941620f23e4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:34.688706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3870bc-5a7c-49ae-a823-2941620f23e4 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:34.701442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-ed3870bc-5a7c-49ae-a823-2941620f23e4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 05 14:14:34.701442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:34.701442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:34.701442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:34.701442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:34.701442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:34.701442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:34.701442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:34.701442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:34.701442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:34.701442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:34.701442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:34.701442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:34.701442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:34.701442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:34.701442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:34.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:34.703568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:34.703568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:34.703568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:34.703568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:34.703568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:34.703568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:34.703568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:34.703568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:34.703568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 05 14:14:34.703568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:34.703568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 250/499] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:34.704686 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3870bc-5a7c-49ae-a823-2941620f23e4 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:34.724703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a25f3-214a-495a-af20-18df0bf10807 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:34.758144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9129d-da57-45a4-835b-1d642f2ae04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:34.758758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9129d-da57-45a4-835b-1d642f2ae04d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:34.759084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9129d-da57-45a4-835b-1d642f2ae04d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:34.759385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9129d-da57-45a4-835b-1d642f2ae04d None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:34.760101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9129d-da57-45a4-835b-1d642f2ae04d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:34.760401 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9129d-da57-45a4-835b-1d642f2ae04d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:34.761831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9129d-da57-45a4-835b-1d642f2ae04d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:34.797131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 250/500] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 5 14:14:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:34.797958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9129d-da57-45a4-835b-1d642f2ae04d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:34.803060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:34.815848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3870bc-5a7c-49ae-a823-2941620f23e4 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:34.852087 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdb2d8-31fe-4043-b757-2e2e0264aca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:34.852714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdb2d8-31fe-4043-b757-2e2e0264aca8 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:34.853019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdb2d8-31fe-4043-b757-2e2e0264aca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:34.853289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdb2d8-31fe-4043-b757-2e2e0264aca8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:34.860829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:34.864264 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdb2d8-31fe-4043-b757-2e2e0264aca8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:34.869446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-29bdb2d8-31fe-4043-b757-2e2e0264aca8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:34.869446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:34.869446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:34.869446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:34.869446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:34.869446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:34.869446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:34.869446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:34.869446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:34.869446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:34.869446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:34.869446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:34.869446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:34.869446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:34.869446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:34.869446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:34.870670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:34.870670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:34.870670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:34.870670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:34.870670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:34.870670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:34.870670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:34.870670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:34.870670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:34.870670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:34.870670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:34.870670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:34.870670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:34.870670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:34.870670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:34.870670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:34.871766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:34.871766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:34.871766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:34.871766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:34.871766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:34.871766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:34.871766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:34.871766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:34.871766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:34.871766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:34.871766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:34.871766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:34.871766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 251/501] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:34.871766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdb2d8-31fe-4043-b757-2e2e0264aca8 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:34.904868 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9129d-da57-45a4-835b-1d642f2ae04d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:34.936838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae731430-ffc7-463d-8ce7-f080b787f759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:34.937453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae731430-ffc7-463d-8ce7-f080b787f759 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:34.937698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae731430-ffc7-463d-8ce7-f080b787f759 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:34.937969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae731430-ffc7-463d-8ce7-f080b787f759 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:34.938617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae731430-ffc7-463d-8ce7-f080b787f759 None admin] RBAC: Adding query filter params (name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:34.938948 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae731430-ffc7-463d-8ce7-f080b787f759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:34.940593 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae731430-ffc7-463d-8ce7-f080b787f759 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:34.948850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 251/502] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:14:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:34.949643 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae731430-ffc7-463d-8ce7-f080b787f759 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:34.970493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdb2d8-31fe-4043-b757-2e2e0264aca8 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:35.003639 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f37b9-d479-4731-a3c4-c72f2b038740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:35.004584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f37b9-d479-4731-a3c4-c72f2b038740 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:35.004956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f37b9-d479-4731-a3c4-c72f2b038740 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:35.005134 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f37b9-d479-4731-a3c4-c72f2b038740 None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:35.005850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f37b9-d479-4731-a3c4-c72f2b038740 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:35.006152 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f37b9-d479-4731-a3c4-c72f2b038740 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:35.007602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f37b9-d479-4731-a3c4-c72f2b038740 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:35.016639 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:35.043874 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 252/503] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Sep 5 14:14:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:35.045086 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718afe1-9677-471a-98a3-9017c1edb631 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:35.045380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718afe1-9677-471a-98a3-9017c1edb631 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:35.045636 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718afe1-9677-471a-98a3-9017c1edb631 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:35.048689 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 253/504] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:35.048689 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718afe1-9677-471a-98a3-9017c1edb631 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:35.054580 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae731430-ffc7-463d-8ce7-f080b787f759 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:35.089407 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e70785-2e1f-4a42-b7a5-2c7cf5c3a29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:35.090011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e70785-2e1f-4a42-b7a5-2c7cf5c3a29e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:35.090296 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e70785-2e1f-4a42-b7a5-2c7cf5c3a29e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:35.090575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e70785-2e1f-4a42-b7a5-2c7cf5c3a29e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:35.109588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:35.150378 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718afe1-9677-471a-98a3-9017c1edb631 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:35.158465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e70785-2e1f-4a42-b7a5-2c7cf5c3a29e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:35.159972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e70785-2e1f-4a42-b7a5-2c7cf5c3a29e None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:35.183570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f5197-93d1-4cb0-bd2e-f260c67b660b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:35.184171 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f5197-93d1-4cb0-bd2e-f260c67b660b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:35.184448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f5197-93d1-4cb0-bd2e-f260c67b660b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:35.184726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f5197-93d1-4cb0-bd2e-f260c67b660b None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:35.185411 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f5197-93d1-4cb0-bd2e-f260c67b660b None admin] RBAC: Adding query filter params (name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:35.185710 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f5197-93d1-4cb0-bd2e-f260c67b660b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:35.187130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f5197-93d1-4cb0-bd2e-f260c67b660b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:35.195140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 254/505] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:14:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:35.195903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f5197-93d1-4cb0-bd2e-f260c67b660b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:35.224534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:35.260226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 252/506] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:35.261039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e70785-2e1f-4a42-b7a5-2c7cf5c3a29e None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:35.293638 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f5197-93d1-4cb0-bd2e-f260c67b660b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:35.313406 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:35.328303 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeaa02-e185-4398-b837-c8ac0c52f195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:35.328919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeaa02-e185-4398-b837-c8ac0c52f195 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:35.329244 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeaa02-e185-4398-b837-c8ac0c52f195 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:35.329517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeaa02-e185-4398-b837-c8ac0c52f195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:35.340256 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbeaa02-e185-4398-b837-c8ac0c52f195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:35.341946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbeaa02-e185-4398-b837-c8ac0c52f195 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:35.356246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-5fbeaa02-e185-4398-b837-c8ac0c52f195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:35.356246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:35.356246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:35.356246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:35.356246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:35.356246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:35.356246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:35.356246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:35.356246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:35.356246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:35.356246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:35.356246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:35.356246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:35.356246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:35.356246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:35.356246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:35.357259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:35.357259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:35.357259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:35.357259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:35.357259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:35.357259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:35.357259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:35.357259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:35.357259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:35.357259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:35.357259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:35.357259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:35.357259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:35.357259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:35.357259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:35.357259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:35.357259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:35.358135 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:35.358135 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:35.358135 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:35.358135 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 255/507] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:35.358889 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4e803-97c5-4c1d-a455-ded3ebe3d99f None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:35.358889 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4e803-97c5-4c1d-a455-ded3ebe3d99f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:35.359046 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4e803-97c5-4c1d-a455-ded3ebe3d99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:35.361453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e70785-2e1f-4a42-b7a5-2c7cf5c3a29e None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:35.376342 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-b3f4e803-97c5-4c1d-a455-ded3ebe3d99f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:35.377629 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b3f4e803-97c5-4c1d-a455-ded3ebe3d99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:35.396615 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080a9a8-e7dd-457f-b256-e419de2af00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:35.397223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080a9a8-e7dd-457f-b256-e419de2af00c None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:35.397511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080a9a8-e7dd-457f-b256-e419de2af00c None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:35.397789 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080a9a8-e7dd-457f-b256-e419de2af00c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:35.429882 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:35.441926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 256/508] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:35.442726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4e803-97c5-4c1d-a455-ded3ebe3d99f None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:35.465490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080a9a8-e7dd-457f-b256-e419de2af00c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:35.467009 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080a9a8-e7dd-457f-b256-e419de2af00c None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:35.552614 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4e803-97c5-4c1d-a455-ded3ebe3d99f None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:35.554136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:35.580981 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 253/509] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:35.586311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080a9a8-e7dd-457f-b256-e419de2af00c None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:35.589752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c2007-2ac6-46eb-9e71-5e9fcd563296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:35.590390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c2007-2ac6-46eb-9e71-5e9fcd563296 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:35.590671 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c2007-2ac6-46eb-9e71-5e9fcd563296 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:35.590984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c2007-2ac6-46eb-9e71-5e9fcd563296 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:35.601600 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c2007-2ac6-46eb-9e71-5e9fcd563296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:35.603085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c2007-2ac6-46eb-9e71-5e9fcd563296 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:35.617663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-0a0c2007-2ac6-46eb-9e71-5e9fcd563296 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:35.617663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:35.617663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:35.617663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:35.617663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:35.617663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:35.617663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:35.617663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:35.617663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:35.617663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:35.617663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:35.617663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:35.617663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:35.617663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:35.617663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:35.617663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:35.618628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:35.618628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:35.618628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:35.618628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:35.618628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:35.618628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:35.618628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:35.618628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:35.618628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:35.618628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:35.618628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:35.618628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:35.618628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:35.618628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:35.618628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:35.618628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:35.618628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:35.619542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:35.619542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:35.619542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:35.619542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 257/510] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:35.619542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c2007-2ac6-46eb-9e71-5e9fcd563296 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:35.659461 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:35.695027 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080a9a8-e7dd-457f-b256-e419de2af00c None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:35.725673 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c2007-2ac6-46eb-9e71-5e9fcd563296 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:35.728998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612921cb-b44d-4166-8c6f-1512b831d428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:35.729622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612921cb-b44d-4166-8c6f-1512b831d428 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:35.729895 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612921cb-b44d-4166-8c6f-1512b831d428 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:35.730163 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612921cb-b44d-4166-8c6f-1512b831d428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:35.740229 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612921cb-b44d-4166-8c6f-1512b831d428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:35.741670 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612921cb-b44d-4166-8c6f-1512b831d428 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:35.755894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-612921cb-b44d-4166-8c6f-1512b831d428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:35.755894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:35.755894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:35.755894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:35.755894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:35.755894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:35.755894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:35.755894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:35.755894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:35.755894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:35.755894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:35.755894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:35.755894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:35.755894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:35.755894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:35.755894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:35.756991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:35.756991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:35.756991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:35.756991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:35.756991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:35.756991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:35.756991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:35.756991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:35.756991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:35.756991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:35.756991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:35.756991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:35.756991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:35.756991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:35.756991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:35.756991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:35.756991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:35.758284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:35.758284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:35.758284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:35.758284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 254/511] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:35.758284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612921cb-b44d-4166-8c6f-1512b831d428 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:35.760690 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e815a-ac67-41a6-b7c0-bfaf7fe44320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:35.762091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e815a-ac67-41a6-b7c0-bfaf7fe44320 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:35.762382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e815a-ac67-41a6-b7c0-bfaf7fe44320 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:35.762659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e815a-ac67-41a6-b7c0-bfaf7fe44320 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:35.763346 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e815a-ac67-41a6-b7c0-bfaf7fe44320 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:35.763657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e815a-ac67-41a6-b7c0-bfaf7fe44320 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:35.765039 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e815a-ac67-41a6-b7c0-bfaf7fe44320 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:35.777562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 258/512] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:35.778342 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e815a-ac67-41a6-b7c0-bfaf7fe44320 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:35.796279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:35.859882 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612921cb-b44d-4166-8c6f-1512b831d428 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:35.876978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:35.879212 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e815a-ac67-41a6-b7c0-bfaf7fe44320 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:35.895215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabd73c-476e-4e0f-9d9a-3468149e1842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:35.895840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabd73c-476e-4e0f-9d9a-3468149e1842 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:35.896115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabd73c-476e-4e0f-9d9a-3468149e1842 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:35.896382 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabd73c-476e-4e0f-9d9a-3468149e1842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:35.907226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfabd73c-476e-4e0f-9d9a-3468149e1842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:35.909312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfabd73c-476e-4e0f-9d9a-3468149e1842 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:35.913136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ccbaf-c201-4713-87b5-9a4dd892650b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:35.913741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ccbaf-c201-4713-87b5-9a4dd892650b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:35.914019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ccbaf-c201-4713-87b5-9a4dd892650b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:35.914288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ccbaf-c201-4713-87b5-9a4dd892650b None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:35.914962 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ccbaf-c201-4713-87b5-9a4dd892650b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:35.915265 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ccbaf-c201-4713-87b5-9a4dd892650b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:35.917205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ccbaf-c201-4713-87b5-9a4dd892650b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:35.924571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-dfabd73c-476e-4e0f-9d9a-3468149e1842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:35.924571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:35.924571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:35.924571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:35.924571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:35.924571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:35.924571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:35.924571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:35.924571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:35.924571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:35.924571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:35.924571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:35.924571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:35.924571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:35.924571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:35.924571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:35.925473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:35.925473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:35.925473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:35.925473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:35.925473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:35.925473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:35.925473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:35.925473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:35.925473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:35.925473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:35.925473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:35.925473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:35.925473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:35.925473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:35.925473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:35.925473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:35.925473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:35.926351 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:35.926351 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:35.926351 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:35.926351 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 255/513] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:35.926886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabd73c-476e-4e0f-9d9a-3468149e1842 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:35.932474 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 259/514] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:35.933384 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ccbaf-c201-4713-87b5-9a4dd892650b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:36.005998 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:36.029801 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabd73c-476e-4e0f-9d9a-3468149e1842 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:36.030289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ccbaf-c201-4713-87b5-9a4dd892650b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:36.063550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b4be7-3c28-4c24-ac9c-695c9c951bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:36.064151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b4be7-3c28-4c24-ac9c-695c9c951bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:36.064250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf355fc-d96a-4537-ad1a-9223ef242775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:36.064432 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b4be7-3c28-4c24-ac9c-695c9c951bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:36.064707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b4be7-3c28-4c24-ac9c-695c9c951bb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:36.064813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf355fc-d96a-4537-ad1a-9223ef242775 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:36.065049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf355fc-d96a-4537-ad1a-9223ef242775 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:36.065573 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf355fc-d96a-4537-ad1a-9223ef242775 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:36.065667 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b4be7-3c28-4c24-ac9c-695c9c951bb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:36.065895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b4be7-3c28-4c24-ac9c-695c9c951bb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:36.067306 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b4be7-3c28-4c24-ac9c-695c9c951bb5 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:36.076423 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf355fc-d96a-4537-ad1a-9223ef242775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:36.077879 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf355fc-d96a-4537-ad1a-9223ef242775 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:36.079697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 260/515] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:36.080490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b4be7-3c28-4c24-ac9c-695c9c951bb5 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:36.099164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-ccf355fc-d96a-4537-ad1a-9223ef242775 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:36.099164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:36.099164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:36.099164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:36.099164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:36.099164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:36.099164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:36.099164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:36.099164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:36.099164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:36.099164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:36.099164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:36.099164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:36.099164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:36.099164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:36.099164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:36.100095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:36.100095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:36.100095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:36.100095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:36.100095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:36.100095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:36.100095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:36.100095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:36.100095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:36.100095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:36.100095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:36.100095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:36.100095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:36.100095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:36.100095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:36.100095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:36.100095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:36.100942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:36.100942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:36.100942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:36.100942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 256/516] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:36.101162 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf355fc-d96a-4537-ad1a-9223ef242775 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:36.125478 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:36.179326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b4be7-3c28-4c24-ac9c-695c9c951bb5 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:36.198117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf355fc-d96a-4537-ad1a-9223ef242775 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:36.214622 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcf47a-fbb6-4f89-93f7-96093d8f95f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:36.215284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcf47a-fbb6-4f89-93f7-96093d8f95f7 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:36.215561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcf47a-fbb6-4f89-93f7-96093d8f95f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:36.215829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcf47a-fbb6-4f89-93f7-96093d8f95f7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:36.217693 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:36.223215 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bcf47a-fbb6-4f89-93f7-96093d8f95f7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:36.224667 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bcf47a-fbb6-4f89-93f7-96093d8f95f7 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:36.232592 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff5d36-af76-4e30-ae1e-af8f19e4f121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:36.232592 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff5d36-af76-4e30-ae1e-af8f19e4f121 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:36.232783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff5d36-af76-4e30-ae1e-af8f19e4f121 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:36.232976 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff5d36-af76-4e30-ae1e-af8f19e4f121 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:36.233610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff5d36-af76-4e30-ae1e-af8f19e4f121 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:36.233913 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff5d36-af76-4e30-ae1e-af8f19e4f121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:36.235299 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff5d36-af76-4e30-ae1e-af8f19e4f121 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:36.235885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-54bcf47a-fbb6-4f89-93f7-96093d8f95f7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 14:14:36.235885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:36.235885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:36.235885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:36.235885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:36.235885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:36.235885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:36.235885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:36.235885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:36.235885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:36.235885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:36.235885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:36.235885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:36.235885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:36.235885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:36.235885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:36.236827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:36.238132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:36.238132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:36.238132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:36.238132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:36.238132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:36.238132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:36.238132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:36.238132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:36.238132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 14:14:36.238132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:36.238132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 261/517] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:36.238132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcf47a-fbb6-4f89-93f7-96093d8f95f7 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:36.251164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 257/518] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:36.252011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff5d36-af76-4e30-ae1e-af8f19e4f121 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:36.335705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:36.338562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcf47a-fbb6-4f89-93f7-96093d8f95f7 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:36.353629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff5d36-af76-4e30-ae1e-af8f19e4f121 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:36.371967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65a9b4-845b-4d12-a295-fcfba9771a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:36.372586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65a9b4-845b-4d12-a295-fcfba9771a3c None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:36.372864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65a9b4-845b-4d12-a295-fcfba9771a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:36.373136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65a9b4-845b-4d12-a295-fcfba9771a3c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:36.380818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65a9b4-845b-4d12-a295-fcfba9771a3c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:36.382292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65a9b4-845b-4d12-a295-fcfba9771a3c None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:36.385815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2072a-8125-4f1e-b3f0-16aaf0e4c7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:36.386692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2072a-8125-4f1e-b3f0-16aaf0e4c7da None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:36.387217 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2072a-8125-4f1e-b3f0-16aaf0e4c7da None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:36.387532 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2072a-8125-4f1e-b3f0-16aaf0e4c7da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:36.393727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-0e65a9b4-845b-4d12-a295-fcfba9771a3c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 14:14:36.393727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:36.393727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:36.393727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:36.393727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:36.393727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:36.393727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:36.393727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:36.393727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:36.393727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:36.393727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:36.393727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:36.393727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:36.393727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:36.393727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:36.393727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:36.394588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:36.395851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:36.395851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:36.395851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:36.395851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:36.395851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:36.395851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:36.395851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:36.395851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:36.395851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 14:14:36.395851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:36.396371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2072a-8125-4f1e-b3f0-16aaf0e4c7da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:36.396441 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 262/519] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:36.396441 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65a9b4-845b-4d12-a295-fcfba9771a3c None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:36.396581 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2072a-8125-4f1e-b3f0-16aaf0e4c7da None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:36.410622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-cfb2072a-8125-4f1e-b3f0-16aaf0e4c7da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:36.410622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:36.410622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:36.410622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:36.410622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:36.410622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:36.410622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:36.410622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:36.410622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:36.410622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:36.410622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:36.410622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:36.410622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:14:36.410622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:14:36.410622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:14:36.411657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:14:36.411657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:36.411657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:36.411657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:36.411657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:36.411657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:36.411657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:36.411657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:36.411657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:36.411657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:36.411657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:36.411657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:36.411657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:36.411657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:36.411657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:36.411657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:36.411657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:36.412487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:14:36.412487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:14:36.412487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:14:36.412487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:14:36.412487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:14:36.412487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:14:36.412487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:36.412487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:36.412487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 258/520] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:36.412994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2072a-8125-4f1e-b3f0-16aaf0e4c7da None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:36.453257 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:36.497461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65a9b4-845b-4d12-a295-fcfba9771a3c None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:36.514933 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2072a-8125-4f1e-b3f0-16aaf0e4c7da None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:36.530036 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588fe0f-8e90-4f8d-a6ed-4d7a9810d1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:36.530631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588fe0f-8e90-4f8d-a6ed-4d7a9810d1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:36.530945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588fe0f-8e90-4f8d-a6ed-4d7a9810d1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:36.531214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588fe0f-8e90-4f8d-a6ed-4d7a9810d1e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:36.531840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588fe0f-8e90-4f8d-a6ed-4d7a9810d1e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:36.532133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588fe0f-8e90-4f8d-a6ed-4d7a9810d1e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:36.533483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588fe0f-8e90-4f8d-a6ed-4d7a9810d1e5 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:36.545493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 263/521] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:36.546256 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588fe0f-8e90-4f8d-a6ed-4d7a9810d1e5 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:36.549272 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97a976-b33f-4aa7-bf3f-d35bc31f819d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:36.549875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97a976-b33f-4aa7-bf3f-d35bc31f819d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:36.550156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97a976-b33f-4aa7-bf3f-d35bc31f819d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:36.550434 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97a976-b33f-4aa7-bf3f-d35bc31f819d None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:36.551193 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97a976-b33f-4aa7-bf3f-d35bc31f819d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:36.551497 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97a976-b33f-4aa7-bf3f-d35bc31f819d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:36.552938 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97a976-b33f-4aa7-bf3f-d35bc31f819d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:36.567619 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:36.570783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 259/522] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Sep 5 14:14:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:36.571640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97a976-b33f-4aa7-bf3f-d35bc31f819d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:36.645148 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588fe0f-8e90-4f8d-a6ed-4d7a9810d1e5 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:36.659657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:36.668123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97a976-b33f-4aa7-bf3f-d35bc31f819d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:36.679688 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc1432-3596-47c1-8fe4-fd2ba98a98e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:36.680314 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc1432-3596-47c1-8fe4-fd2ba98a98e2 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:36.680587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc1432-3596-47c1-8fe4-fd2ba98a98e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:36.680861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc1432-3596-47c1-8fe4-fd2ba98a98e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:36.687833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbc1432-3596-47c1-8fe4-fd2ba98a98e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:36.689316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbc1432-3596-47c1-8fe4-fd2ba98a98e2 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:36.703168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-cdbc1432-3596-47c1-8fe4-fd2ba98a98e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:36.703168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:36.703168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:36.703168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:36.703168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:36.703168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:36.703168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:36.703168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:36.703168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:36.703168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:36.703168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:36.703168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:36.703168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:14:36.703168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:14:36.703168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:14:36.704043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:14:36.704043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:36.704043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:36.704043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:36.704043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:36.704043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:36.704043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:36.704043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:36.704043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:36.704043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:36.704043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:36.704043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:36.704043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:36.704043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:36.704043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:36.704043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:36.704043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:36.704894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:14:36.704894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:14:36.704894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:14:36.704894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:14:36.704894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:14:36.704894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:14:36.704894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:36.704894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:36.704894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 264/523] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:36.705361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7175d-47c4-4dc3-8aad-a4e162011ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:36.705361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7175d-47c4-4dc3-8aad-a4e162011ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:36.705510 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc1432-3596-47c1-8fe4-fd2ba98a98e2 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:36.705579 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7175d-47c4-4dc3-8aad-a4e162011ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:36.705579 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7175d-47c4-4dc3-8aad-a4e162011ae3 None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:36.706237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7175d-47c4-4dc3-8aad-a4e162011ae3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:36.706466 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7175d-47c4-4dc3-8aad-a4e162011ae3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:36.707949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7175d-47c4-4dc3-8aad-a4e162011ae3 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:36.725925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 260/524] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Sep 5 14:14:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:36.726729 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7175d-47c4-4dc3-8aad-a4e162011ae3 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:36.785460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:36.809602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc1432-3596-47c1-8fe4-fd2ba98a98e2 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:36.826899 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7175d-47c4-4dc3-8aad-a4e162011ae3 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:36.844752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a0f32-1ed1-4c17-bae7-1fc3a8abba5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:36.845358 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a0f32-1ed1-4c17-bae7-1fc3a8abba5c None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:36.845639 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a0f32-1ed1-4c17-bae7-1fc3a8abba5c None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:36.845913 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a0f32-1ed1-4c17-bae7-1fc3a8abba5c None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:36.846599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a0f32-1ed1-4c17-bae7-1fc3a8abba5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:36.846930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a0f32-1ed1-4c17-bae7-1fc3a8abba5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:36.848356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a0f32-1ed1-4c17-bae7-1fc3a8abba5c None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:36.859736 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe3e11-2607-4b6f-a79e-c7d1734a2cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:36.860357 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe3e11-2607-4b6f-a79e-c7d1734a2cf5 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:36.865255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe3e11-2607-4b6f-a79e-c7d1734a2cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:36.865255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe3e11-2607-4b6f-a79e-c7d1734a2cf5 None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:36.865255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe3e11-2607-4b6f-a79e-c7d1734a2cf5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:36.865255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe3e11-2607-4b6f-a79e-c7d1734a2cf5 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:36.865789 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 265/525] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:14:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:36.866579 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a0f32-1ed1-4c17-bae7-1fc3a8abba5c None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:36.900486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:37.014161 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a0f32-1ed1-4c17-bae7-1fc3a8abba5c None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:37.020827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 261/526] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:14:36 2022] POST /identity/v3/users => generated 247 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:14:37.021649 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe3e11-2607-4b6f-a79e-c7d1734a2cf5 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:37.065214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5006b-420c-4e83-b247-aef8e40d5a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:37.067034 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5006b-420c-4e83-b247-aef8e40d5a72 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:37.067324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5006b-420c-4e83-b247-aef8e40d5a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:37.067601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5006b-420c-4e83-b247-aef8e40d5a72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:37.142289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe3e11-2607-4b6f-a79e-c7d1734a2cf5 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:37.148661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5006b-420c-4e83-b247-aef8e40d5a72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:37.150317 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5006b-420c-4e83-b247-aef8e40d5a72 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:37.159819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:37.181117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbca9f9-3ecd-45be-802a-4dd03c3751b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:37.181810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbca9f9-3ecd-45be-802a-4dd03c3751b9 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:37.182692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbca9f9-3ecd-45be-802a-4dd03c3751b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:37.182692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbca9f9-3ecd-45be-802a-4dd03c3751b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:37.183224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbca9f9-3ecd-45be-802a-4dd03c3751b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:37.183529 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbca9f9-3ecd-45be-802a-4dd03c3751b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:37.184971 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbca9f9-3ecd-45be-802a-4dd03c3751b9 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:37.190886 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:37.205318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 262/527] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:14:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:37.206159 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbca9f9-3ecd-45be-802a-4dd03c3751b9 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:37.268032 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 266/528] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:37.269529 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5006b-420c-4e83-b247-aef8e40d5a72 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:37.316885 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbca9f9-3ecd-45be-802a-4dd03c3751b9 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:37.351055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa746f32-71e4-4ee5-8037-ecd7e2decc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:37.351668 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa746f32-71e4-4ee5-8037-ecd7e2decc84 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:37.351949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa746f32-71e4-4ee5-8037-ecd7e2decc84 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:37.352224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa746f32-71e4-4ee5-8037-ecd7e2decc84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:37.375681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5006b-420c-4e83-b247-aef8e40d5a72 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:37.395569 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:37.411599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d002e-830c-469b-954f-f6144d96e024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:37.412486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d002e-830c-469b-954f-f6144d96e024 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:37.412941 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d002e-830c-469b-954f-f6144d96e024 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:37.413386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d002e-830c-469b-954f-f6144d96e024 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:37.414301 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d002e-830c-469b-954f-f6144d96e024 None admin] RBAC: Adding query filter params (role.id=0a83e40e2d64470a8489bf548e34dabd, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=709b9ae82dcd47408fc9a6d54a54428a) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:37.414782 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d002e-830c-469b-954f-f6144d96e024 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:37.416488 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d002e-830c-469b-954f-f6144d96e024 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:37.418444 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:37.422516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa746f32-71e4-4ee5-8037-ecd7e2decc84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:37.423241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 267/529] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:14:37 2022] GET /identity/v3/role_assignments?user.id=709b9ae82dcd47408fc9a6d54a54428a&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=0a83e40e2d64470a8489bf548e34dabd => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:37.424159 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa746f32-71e4-4ee5-8037-ecd7e2decc84 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:37.424261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d002e-830c-469b-954f-f6144d96e024 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:37.553650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d002e-830c-469b-954f-f6144d96e024 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:37.582535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 263/530] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:37.582535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa746f32-71e4-4ee5-8037-ecd7e2decc84 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:37.607758 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85419657-63a6-4c7c-915a-d1aff5beef5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:37.608404 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85419657-63a6-4c7c-915a-d1aff5beef5a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:37.608690 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85419657-63a6-4c7c-915a-d1aff5beef5a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:37.608959 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85419657-63a6-4c7c-915a-d1aff5beef5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:37.609682 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85419657-63a6-4c7c-915a-d1aff5beef5a None admin] RBAC: Adding query filter params (role.id=0a83e40e2d64470a8489bf548e34dabd, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=735986478c354ed196c601f12f3ac377) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:37.609984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85419657-63a6-4c7c-915a-d1aff5beef5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:37.611980 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85419657-63a6-4c7c-915a-d1aff5beef5a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:37.619479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 268/531] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:14:37 2022] GET /identity/v3/role_assignments?user.id=735986478c354ed196c601f12f3ac377&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=0a83e40e2d64470a8489bf548e34dabd => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:37.620592 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85419657-63a6-4c7c-915a-d1aff5beef5a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:37.688386 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:37.690300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:37.743216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa746f32-71e4-4ee5-8037-ecd7e2decc84 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:37.780851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85419657-63a6-4c7c-915a-d1aff5beef5a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:37.792842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa89aa-9f98-4efa-a2a1-ba937a1ca560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:37.793924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa89aa-9f98-4efa-a2a1-ba937a1ca560 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:37.794451 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa89aa-9f98-4efa-a2a1-ba937a1ca560 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:37.794955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa89aa-9f98-4efa-a2a1-ba937a1ca560 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:37.807008 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa89aa-9f98-4efa-a2a1-ba937a1ca560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:37.808838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa89aa-9f98-4efa-a2a1-ba937a1ca560 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:37.827443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-6afa89aa-9f98-4efa-a2a1-ba937a1ca560 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:37.827443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:37.827443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:37.827443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:37.827443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:37.827443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:37.827443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:37.827443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:37.827443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:37.827443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:37.827443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:37.827443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:37.827443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:37.827443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:37.827443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:37.827443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:37.828448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:37.828448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:37.828448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:37.828448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:37.828448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:37.828448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:37.828448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:37.828448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:37.828448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:37.828448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:37.828448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:37.828448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:37.828448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:37.828448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:37.828448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:37.828448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:37.828448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:37.829314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:37.829314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:37.829314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:37.829314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 264/532] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:37.836059 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa89aa-9f98-4efa-a2a1-ba937a1ca560 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:37.855026 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623aaf90-924a-453f-8d62-fce35727586b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:37.856276 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623aaf90-924a-453f-8d62-fce35727586b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:37.856759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623aaf90-924a-453f-8d62-fce35727586b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:37.857471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623aaf90-924a-453f-8d62-fce35727586b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:37.871769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623aaf90-924a-453f-8d62-fce35727586b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:37.874152 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623aaf90-924a-453f-8d62-fce35727586b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:37.892349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-623aaf90-924a-453f-8d62-fce35727586b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:37.892349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:37.892349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:37.892349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:37.892349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:37.892349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:37.892349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:37.892349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:37.892349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:37.892349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:37.892349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:37.892349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:37.892349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:37.892349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:37.892349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:37.892349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:37.893320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:37.893320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:37.893320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:37.893320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:37.893320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:37.893320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:37.893320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:37.893320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:37.893320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:37.893320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:37.893320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:37.893320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:37.893320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:37.893320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:37.893320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:37.893320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:37.893320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:37.894170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:37.894170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:37.894170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:37.896741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 269/533] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:37.902545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623aaf90-924a-453f-8d62-fce35727586b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:38.001021 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa89aa-9f98-4efa-a2a1-ba937a1ca560 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:38.042705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:38.054194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf815d93-9b21-4704-8123-abee2b2058e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:38.063128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf815d93-9b21-4704-8123-abee2b2058e6 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:38.063128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf815d93-9b21-4704-8123-abee2b2058e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:38.063128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf815d93-9b21-4704-8123-abee2b2058e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:38.063128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf815d93-9b21-4704-8123-abee2b2058e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:38.063128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf815d93-9b21-4704-8123-abee2b2058e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:38.063128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf815d93-9b21-4704-8123-abee2b2058e6 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:38.070702 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 265/534] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:38.076622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf815d93-9b21-4704-8123-abee2b2058e6 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:38.094741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623aaf90-924a-453f-8d62-fce35727586b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:38.109854 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:38.154269 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d704b06-8bb2-4120-be17-bb607b32edef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:38.154917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d704b06-8bb2-4120-be17-bb607b32edef None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:38.155201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d704b06-8bb2-4120-be17-bb607b32edef None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:38.155473 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d704b06-8bb2-4120-be17-bb607b32edef None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:38.156121 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d704b06-8bb2-4120-be17-bb607b32edef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:38.156422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d704b06-8bb2-4120-be17-bb607b32edef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:38.157808 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d704b06-8bb2-4120-be17-bb607b32edef None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:38.170642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 270/535] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:38.179741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d704b06-8bb2-4120-be17-bb607b32edef None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:38.190114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf815d93-9b21-4704-8123-abee2b2058e6 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:38.236547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefd363-8afc-42ab-bc02-fcdaacec03a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:38.237191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefd363-8afc-42ab-bc02-fcdaacec03a1 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:38.237469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefd363-8afc-42ab-bc02-fcdaacec03a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:38.237741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefd363-8afc-42ab-bc02-fcdaacec03a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:38.245679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefd363-8afc-42ab-bc02-fcdaacec03a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:38.247182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefd363-8afc-42ab-bc02-fcdaacec03a1 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:38.258838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-aaefd363-8afc-42ab-bc02-fcdaacec03a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:38.258838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:38.258838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:38.258838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:38.258838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:38.258838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:38.258838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:38.258838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:38.258838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:38.258838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:38.258838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:38.258838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:38.258838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:14:38.258838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:14:38.258838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:14:38.260972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:14:38.260972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:38.260972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:38.260972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:38.260972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:38.260972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:38.260972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:38.260972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:38.260972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:38.260972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:38.260972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:38.260972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:38.260972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:38.260972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:38.260972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:38.260972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:38.260972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:38.263013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:14:38.263013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:14:38.263013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:14:38.263013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:14:38.263013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:14:38.263013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:14:38.263013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:38.263013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:38.263013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 266/536] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:14:38.265794 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefd363-8afc-42ab-bc02-fcdaacec03a1 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:38.319570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:38.374825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefd363-8afc-42ab-bc02-fcdaacec03a1 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:38.409816 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d704b06-8bb2-4120-be17-bb607b32edef None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:38.436822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbff05-2ef4-4751-9c03-d678a9f350ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:38.437763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbff05-2ef4-4751-9c03-d678a9f350ae None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:38.438235 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbff05-2ef4-4751-9c03-d678a9f350ae None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:38.438649 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbff05-2ef4-4751-9c03-d678a9f350ae None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:38.439515 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbff05-2ef4-4751-9c03-d678a9f350ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:38.439970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbff05-2ef4-4751-9c03-d678a9f350ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:38.441801 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbff05-2ef4-4751-9c03-d678a9f350ae None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:38.448951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02a17f-3c18-4a62-ac0a-71fcd004c343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:38.449694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02a17f-3c18-4a62-ac0a-71fcd004c343 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:38.450071 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02a17f-3c18-4a62-ac0a-71fcd004c343 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:38.450454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02a17f-3c18-4a62-ac0a-71fcd004c343 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:38.457814 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02a17f-3c18-4a62-ac0a-71fcd004c343 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:38.459399 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02a17f-3c18-4a62-ac0a-71fcd004c343 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:38.467187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 267/537] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:14:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:38.476383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbff05-2ef4-4751-9c03-d678a9f350ae None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:38.476750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-5a02a17f-3c18-4a62-ac0a-71fcd004c343 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:38.476750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:38.476750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:38.476750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:38.476750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:38.476750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:38.476750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:38.476750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:38.476750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:38.476750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:38.476750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:38.476750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:38.476750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:14:38.476750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:14:38.476750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:14:38.477792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:14:38.477792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:38.477792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:38.477792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:38.477792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:38.477792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:38.477792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:38.477792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:38.477792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:38.477792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:38.477792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:38.477792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:38.477792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:38.477792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:38.477792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:38.477792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:38.477792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:38.478960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:14:38.478960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:14:38.478960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:14:38.478960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:14:38.478960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:14:38.478960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:14:38.478960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:38.478960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:38.480562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 271/538] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:38.490463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02a17f-3c18-4a62-ac0a-71fcd004c343 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:38.552174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:38.623264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbff05-2ef4-4751-9c03-d678a9f350ae None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:38.659266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c4b69-a8ac-4d8d-a650-dd2bdc0f1044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:38.659893 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c4b69-a8ac-4d8d-a650-dd2bdc0f1044 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:38.660178 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c4b69-a8ac-4d8d-a650-dd2bdc0f1044 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:38.660456 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c4b69-a8ac-4d8d-a650-dd2bdc0f1044 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:38.661199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c4b69-a8ac-4d8d-a650-dd2bdc0f1044 None admin] RBAC: Adding query filter params (role.id=0a83e40e2d64470a8489bf548e34dabd, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=e49ac59302cb42f7942954d3859d3675) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:38.661512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c4b69-a8ac-4d8d-a650-dd2bdc0f1044 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:38.663281 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c4b69-a8ac-4d8d-a650-dd2bdc0f1044 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:38.665663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02a17f-3c18-4a62-ac0a-71fcd004c343 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:38.669628 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 268/539] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:14:38 2022] GET /identity/v3/role_assignments?user.id=e49ac59302cb42f7942954d3859d3675&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=0a83e40e2d64470a8489bf548e34dabd => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:38.670353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:38.740503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c805dbd-5d15-4551-bf7d-e947a6d9b45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:38.741215 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c805dbd-5d15-4551-bf7d-e947a6d9b45b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:38.741421 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c805dbd-5d15-4551-bf7d-e947a6d9b45b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:38.741694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c805dbd-5d15-4551-bf7d-e947a6d9b45b None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:38.742410 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c805dbd-5d15-4551-bf7d-e947a6d9b45b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:38.742718 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c805dbd-5d15-4551-bf7d-e947a6d9b45b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:38.744338 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c805dbd-5d15-4551-bf7d-e947a6d9b45b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:38.766544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 272/540] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:14:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:38.772731 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c4b69-a8ac-4d8d-a650-dd2bdc0f1044 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:38.907072 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c4b69-a8ac-4d8d-a650-dd2bdc0f1044 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:38.942004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9675e-b2be-4bb6-8751-bc25d2d7ffbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:38.942627 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9675e-b2be-4bb6-8751-bc25d2d7ffbf None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:38.942930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9675e-b2be-4bb6-8751-bc25d2d7ffbf None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:38.943207 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9675e-b2be-4bb6-8751-bc25d2d7ffbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:38.944009 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9675e-b2be-4bb6-8751-bc25d2d7ffbf None admin] RBAC: Adding query filter params (role.id=0a83e40e2d64470a8489bf548e34dabd, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=5c73ca2785334fc496b5227d3be655c7) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:38.944322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9675e-b2be-4bb6-8751-bc25d2d7ffbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:38.945766 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9675e-b2be-4bb6-8751-bc25d2d7ffbf None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:38.952525 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 269/541] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:14:38 2022] GET /identity/v3/role_assignments?user.id=5c73ca2785334fc496b5227d3be655c7&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=0a83e40e2d64470a8489bf548e34dabd => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:39.294222 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fadd7-6b38-4523-b3e8-6be9fcf19cb4 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:39.294611 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fadd7-6b38-4523-b3e8-6be9fcf19cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:39.294797 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fadd7-6b38-4523-b3e8-6be9fcf19cb4 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:39.296396 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 273/542] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:39.301750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c2027-80e2-4a65-b725-08a8628e2610 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:39.302188 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c2027-80e2-4a65-b725-08a8628e2610 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:39.302361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c2027-80e2-4a65-b725-08a8628e2610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:39.320778 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-850c2027-80e2-4a65-b725-08a8628e2610 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:39.322518 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-850c2027-80e2-4a65-b725-08a8628e2610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:39.349551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:39.391346 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 270/543] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:14:39.546397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7a540-b0f1-45d7-8650-5ea92181b6c5 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:39.546763 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7a540-b0f1-45d7-8650-5ea92181b6c5 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:39.547154 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7a540-b0f1-45d7-8650-5ea92181b6c5 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:39.548589 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 274/544] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:39.553789 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98a8af-7c12-424f-809e-32639b1a17ec None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:39.554075 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98a8af-7c12-424f-809e-32639b1a17ec None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:39.554348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98a8af-7c12-424f-809e-32639b1a17ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:39.573509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-bb98a8af-7c12-424f-809e-32639b1a17ec None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:39.574683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-bb98a8af-7c12-424f-809e-32639b1a17ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:39.642230 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 271/545] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:39.876360 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7a540-b0f1-45d7-8650-5ea92181b6c5 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:39.952443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98a8af-7c12-424f-809e-32639b1a17ec None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:39.991660 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:40.000913 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7a540-b0f1-45d7-8650-5ea92181b6c5 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:40.038075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c315075-2db4-4c0b-87c0-434a345f8272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:40.038710 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c315075-2db4-4c0b-87c0-434a345f8272 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:40.039014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c315075-2db4-4c0b-87c0-434a345f8272 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:40.039291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c315075-2db4-4c0b-87c0-434a345f8272 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:40.051936 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c315075-2db4-4c0b-87c0-434a345f8272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:40.056944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-0c315075-2db4-4c0b-87c0-434a345f8272 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:40.056944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:40.056944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:40.056944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:40.056944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:40.056944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:40.056944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:40.056944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:40.056944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:40.056944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:40.056944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:40.056944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:40.056944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:40.056944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:40.056944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:40.056944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:40.057840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:40.057840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:40.057840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:40.057840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:40.057840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:40.057840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:40.057840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:40.057840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:40.057840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:40.057840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:40.057840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:40.057840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:40.057840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:40.057840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:40.057840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:40.057840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:40.058633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:40.058633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:40.058633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:40.058633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:40.058633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:40.058633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:40.058633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:40.058633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:40.058633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:40.058633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:40.058633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:40.058633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:40.058633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 275/546] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:40.064507 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c315075-2db4-4c0b-87c0-434a345f8272 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:40.108712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:40.116016 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98a8af-7c12-424f-809e-32639b1a17ec None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:40.152851 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2b904-973d-463b-8336-86051211efca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:40.153408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2b904-973d-463b-8336-86051211efca None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:40.153690 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2b904-973d-463b-8336-86051211efca None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:40.153962 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2b904-973d-463b-8336-86051211efca None admin] PATH_INFO: `/v3/roles/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:40.164841 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba2b904-973d-463b-8336-86051211efca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:40.170174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-0ba2b904-973d-463b-8336-86051211efca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:40.170174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:40.170174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:40.170174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:40.170174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:40.170174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:40.170174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:40.170174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:40.170174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:40.170174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:40.170174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:40.170174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:40.170174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:40.170174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:40.170174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:40.170174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:40.171104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:40.171104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:40.171104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:40.171104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:40.171104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:40.171104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:40.171104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:40.171104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:40.171104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:40.171104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:40.171104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:40.171104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:40.171104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:40.171104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:40.171104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:40.171104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:40.171988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:40.171988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:40.171988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:40.171988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:40.171988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:40.171988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:40.171988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:40.171988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:40.171988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:40.171988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:40.171988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:40.171988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:40.171988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 272/547] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:40.173109 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7d221-6df3-4a07-bfd3-1c6cc37f5f66 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:40.173653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7d221-6df3-4a07-bfd3-1c6cc37f5f66 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:40.173653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7d221-6df3-4a07-bfd3-1c6cc37f5f66 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:40.178762 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 273/548] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:40.179605 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7d221-6df3-4a07-bfd3-1c6cc37f5f66 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:40.191896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c315075-2db4-4c0b-87c0-434a345f8272 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:40.242136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87f26a-c8d2-4644-b7eb-dea5a08c86b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:40.243451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87f26a-c8d2-4644-b7eb-dea5a08c86b0 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:40.243973 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87f26a-c8d2-4644-b7eb-dea5a08c86b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:40.244860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87f26a-c8d2-4644-b7eb-dea5a08c86b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:40.246023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87f26a-c8d2-4644-b7eb-dea5a08c86b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:40.246762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87f26a-c8d2-4644-b7eb-dea5a08c86b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:40.249077 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87f26a-c8d2-4644-b7eb-dea5a08c86b0 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:40.265590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 276/549] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:14:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:40.265590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5180a-b525-454e-b5bf-55f1e7ecbfb2 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:40.265590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5180a-b525-454e-b5bf-55f1e7ecbfb2 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:40.266933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5180a-b525-454e-b5bf-55f1e7ecbfb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:40.295290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-97e5180a-b525-454e-b5bf-55f1e7ecbfb2 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:40.296461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-97e5180a-b525-454e-b5bf-55f1e7ecbfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:40.300851 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7d221-6df3-4a07-bfd3-1c6cc37f5f66 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:40.308361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:40.344334 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4b394-f48f-4cb5-952e-ea326827f22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:40.344985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4b394-f48f-4cb5-952e-ea326827f22d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:40.345271 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4b394-f48f-4cb5-952e-ea326827f22d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:40.345709 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4b394-f48f-4cb5-952e-ea326827f22d None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:40.346430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4b394-f48f-4cb5-952e-ea326827f22d None admin] RBAC: Adding query filter params (name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:40.346740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4b394-f48f-4cb5-952e-ea326827f22d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:40.348971 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4b394-f48f-4cb5-952e-ea326827f22d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:40.358536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 274/550] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:14:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:40.359435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4b394-f48f-4cb5-952e-ea326827f22d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:40.381832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:40.394399 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 277/551] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:40.394399 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5180a-b525-454e-b5bf-55f1e7ecbfb2 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:40.544517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5180a-b525-454e-b5bf-55f1e7ecbfb2 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:40.562499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4b394-f48f-4cb5-952e-ea326827f22d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:40.583596 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26c39b-d31f-4c8b-8fd4-6681028a6fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:40.584347 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26c39b-d31f-4c8b-8fd4-6681028a6fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:40.584625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26c39b-d31f-4c8b-8fd4-6681028a6fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:40.584892 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26c39b-d31f-4c8b-8fd4-6681028a6fe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:40.607653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26c39b-d31f-4c8b-8fd4-6681028a6fe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:40.609148 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26c39b-d31f-4c8b-8fd4-6681028a6fe0 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:40.622630 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:40.631945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-ca26c39b-d31f-4c8b-8fd4-6681028a6fe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:40.631945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:40.631945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:40.631945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:40.631945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:40.631945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:40.631945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:40.631945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:40.631945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:40.631945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:40.631945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:40.631945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:40.631945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:40.631945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:40.631945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:40.631945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:40.632871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:40.632871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:40.632871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:40.632871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:40.632871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:40.632871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:40.632871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:40.632871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:40.632871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:40.632871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:40.632871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:40.632871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:40.632871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:40.632871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:40.632871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:40.632871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:40.632871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:40.633906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:40.633906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:40.633906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:40.633906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 278/552] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:40.639335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26c39b-d31f-4c8b-8fd4-6681028a6fe0 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:40.647888 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd76c75-491e-4e9d-9158-a1a3d8ab07c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:40.648597 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd76c75-491e-4e9d-9158-a1a3d8ab07c1 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:40.648879 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd76c75-491e-4e9d-9158-a1a3d8ab07c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:40.649151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd76c75-491e-4e9d-9158-a1a3d8ab07c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:40.667859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd76c75-491e-4e9d-9158-a1a3d8ab07c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:40.670129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd76c75-491e-4e9d-9158-a1a3d8ab07c1 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:40.695942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-bcd76c75-491e-4e9d-9158-a1a3d8ab07c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:40.695942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:40.695942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:40.695942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:40.695942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:40.695942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:40.695942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:40.695942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:40.695942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:40.695942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:40.695942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:40.695942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:40.695942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:40.695942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:40.695942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:40.695942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:40.697003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:40.697003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:40.697003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:40.697003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:40.697003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:40.697003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:40.697003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:40.697003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:40.697003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:40.697003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:40.697003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:40.697003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:40.697003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:40.697003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:40.697003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:40.697003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:40.697003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:40.699533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:40.699533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:40.699533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:40.699533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 275/553] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:40.703432 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd76c75-491e-4e9d-9158-a1a3d8ab07c1 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:40.777124 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26c39b-d31f-4c8b-8fd4-6681028a6fe0 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:40.784905 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:40.814843 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aedc8b-81a9-4a09-a780-30314e36dade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:40.815986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aedc8b-81a9-4a09-a780-30314e36dade None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:40.816601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aedc8b-81a9-4a09-a780-30314e36dade None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:40.817048 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aedc8b-81a9-4a09-a780-30314e36dade None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:40.818525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aedc8b-81a9-4a09-a780-30314e36dade None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:40.819029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aedc8b-81a9-4a09-a780-30314e36dade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:40.821043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aedc8b-81a9-4a09-a780-30314e36dade None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:40.827291 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd76c75-491e-4e9d-9158-a1a3d8ab07c1 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:40.838477 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 279/554] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:40.839645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aedc8b-81a9-4a09-a780-30314e36dade None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:40.870684 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda867d2-ee78-4638-a6f4-53fe120e2805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:40.871318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda867d2-ee78-4638-a6f4-53fe120e2805 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:40.871657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda867d2-ee78-4638-a6f4-53fe120e2805 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:40.872020 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda867d2-ee78-4638-a6f4-53fe120e2805 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:40.872548 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda867d2-ee78-4638-a6f4-53fe120e2805 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:40.872847 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda867d2-ee78-4638-a6f4-53fe120e2805 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:40.874241 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda867d2-ee78-4638-a6f4-53fe120e2805 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:40.891149 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 276/555] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:40.892080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda867d2-ee78-4638-a6f4-53fe120e2805 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:40.917508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:40.972027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aedc8b-81a9-4a09-a780-30314e36dade None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:41.001718 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda867d2-ee78-4638-a6f4-53fe120e2805 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:41.014720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5449c-0d5d-4a0a-9f94-0ffa64746d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:41.015391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5449c-0d5d-4a0a-9f94-0ffa64746d9a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:41.015672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5449c-0d5d-4a0a-9f94-0ffa64746d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:41.015995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5449c-0d5d-4a0a-9f94-0ffa64746d9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:41.027045 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5449c-0d5d-4a0a-9f94-0ffa64746d9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:41.031785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-bfa5449c-0d5d-4a0a-9f94-0ffa64746d9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:41.031785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:41.031785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:41.031785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:41.031785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:41.031785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:41.031785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:41.031785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:41.031785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:41.031785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:41.031785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:41.031785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:41.031785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:41.031785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:41.031785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:41.031785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:41.032709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:41.032709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:41.032709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:41.032709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:41.032709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:41.032709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:41.032709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:41.032709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:41.032709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:41.032709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:41.032709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:41.032709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:41.032709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:41.032709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:41.032709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:41.032709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:41.033561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:41.033561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:41.033561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:41.033561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:41.033561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:41.033561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:41.033561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:41.033561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:41.033561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:41.033561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:41.033561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:41.033561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:41.033561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 280/556] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:41.034322 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5449c-0d5d-4a0a-9f94-0ffa64746d9a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:41.035188 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:41.048224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302064a-e5c7-4dc7-bdb7-badafd107381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:41.049131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302064a-e5c7-4dc7-bdb7-badafd107381 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:41.055763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302064a-e5c7-4dc7-bdb7-badafd107381 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:41.055763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302064a-e5c7-4dc7-bdb7-badafd107381 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:41.058162 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302064a-e5c7-4dc7-bdb7-badafd107381 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:41.059734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302064a-e5c7-4dc7-bdb7-badafd107381 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:41.074727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-6302064a-e5c7-4dc7-bdb7-badafd107381 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:14:41.074727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:41.074727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:41.074727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:41.074727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:41.074727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:41.074727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:41.074727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:41.074727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:41.074727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:41.074727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:41.074727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:41.074727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:41.074727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:41.074727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:41.074727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:41.076161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:41.077112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:41.077112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:41.077112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:41.077112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:41.077112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:41.077112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:41.077112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:41.077112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:41.077112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:14:41.077112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:41.080408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 277/557] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 5 14:14:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:41.081727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77b6e3-c740-4aa1-a293-1ba308869c40 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:41.082008 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77b6e3-c740-4aa1-a293-1ba308869c40 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:41.082282 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77b6e3-c740-4aa1-a293-1ba308869c40 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:41.088318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 278/558] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:41.088318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f77b6e3-c740-4aa1-a293-1ba308869c40 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:41.156858 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5449c-0d5d-4a0a-9f94-0ffa64746d9a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:41.176386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:41.190062 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f77b6e3-c740-4aa1-a293-1ba308869c40 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:41.193955 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d180fc-13c2-48fc-b5e4-c32296d0d2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:41.194584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d180fc-13c2-48fc-b5e4-c32296d0d2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:41.194891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d180fc-13c2-48fc-b5e4-c32296d0d2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:41.195165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d180fc-13c2-48fc-b5e4-c32296d0d2b5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:41.202418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d180fc-13c2-48fc-b5e4-c32296d0d2b5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:41.204089 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d180fc-13c2-48fc-b5e4-c32296d0d2b5 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:41.217628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-82d180fc-13c2-48fc-b5e4-c32296d0d2b5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 14:14:41.217628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:41.217628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:41.217628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:41.217628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:41.217628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:41.217628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:41.217628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:41.217628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:41.217628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:41.217628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:41.217628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:41.217628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:41.217628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:41.217628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:41.217628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:41.218666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:41.220223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:41.220223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:41.220223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:41.220223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:41.220223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:41.220223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:41.220223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:41.220223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:41.220223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 14:14:41.220223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:41.220223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 281/559] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:14:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:41.220223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d180fc-13c2-48fc-b5e4-c32296d0d2b5 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:41.230304 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c8fb5-6ebc-40c9-ae27-30cda2924524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:41.230943 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c8fb5-6ebc-40c9-ae27-30cda2924524 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:41.231224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c8fb5-6ebc-40c9-ae27-30cda2924524 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:41.231495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c8fb5-6ebc-40c9-ae27-30cda2924524 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:41.232151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c8fb5-6ebc-40c9-ae27-30cda2924524 None admin] RBAC: Adding query filter params (name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:41.232452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c8fb5-6ebc-40c9-ae27-30cda2924524 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:41.233824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c8fb5-6ebc-40c9-ae27-30cda2924524 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:41.241793 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 279/560] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:14:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:41.242956 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca862f-4583-4736-9f17-d030f8b99cc7 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:41.243236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca862f-4583-4736-9f17-d030f8b99cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:41.244032 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca862f-4583-4736-9f17-d030f8b99cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:41.261591 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-15ca862f-4583-4736-9f17-d030f8b99cc7 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:41.262809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-15ca862f-4583-4736-9f17-d030f8b99cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:41.308788 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:41.326862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d180fc-13c2-48fc-b5e4-c32296d0d2b5 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:41.328991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 280/561] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:41.329807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca862f-4583-4736-9f17-d030f8b99cc7 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:41.362360 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeabcc1-b783-46f9-8135-105da704babc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:41.362996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeabcc1-b783-46f9-8135-105da704babc None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:41.363285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeabcc1-b783-46f9-8135-105da704babc None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:41.363551 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeabcc1-b783-46f9-8135-105da704babc None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:41.364281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeabcc1-b783-46f9-8135-105da704babc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:41.364588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeabcc1-b783-46f9-8135-105da704babc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:41.366007 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeabcc1-b783-46f9-8135-105da704babc None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:41.383336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 282/562] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 5 14:14:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:41.388237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b036ad8-cacb-46b4-9027-74954de27e55 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:41.388237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b036ad8-cacb-46b4-9027-74954de27e55 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:41.388237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b036ad8-cacb-46b4-9027-74954de27e55 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:41.388237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 283/563] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:41.388237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b036ad8-cacb-46b4-9027-74954de27e55 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:41.425732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:41.441820 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca862f-4583-4736-9f17-d030f8b99cc7 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:41.476924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46eeec4-6fda-4fc0-91de-d3140ad9161d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:41.477535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46eeec4-6fda-4fc0-91de-d3140ad9161d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:41.477898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46eeec4-6fda-4fc0-91de-d3140ad9161d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:41.478165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46eeec4-6fda-4fc0-91de-d3140ad9161d None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:41.478919 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46eeec4-6fda-4fc0-91de-d3140ad9161d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:41.479229 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46eeec4-6fda-4fc0-91de-d3140ad9161d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:41.480740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46eeec4-6fda-4fc0-91de-d3140ad9161d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:41.494939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b036ad8-cacb-46b4-9027-74954de27e55 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:41.502774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 281/564] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Sep 5 14:14:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:41.502774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46eeec4-6fda-4fc0-91de-d3140ad9161d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:41.530729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39c3f1-8e6f-4111-9298-95b11e8acfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:41.531377 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39c3f1-8e6f-4111-9298-95b11e8acfe9 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:41.531657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39c3f1-8e6f-4111-9298-95b11e8acfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:41.531930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39c3f1-8e6f-4111-9298-95b11e8acfe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:41.541944 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:41.542921 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39c3f1-8e6f-4111-9298-95b11e8acfe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:41.544398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39c3f1-8e6f-4111-9298-95b11e8acfe9 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:41.559001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-ba39c3f1-8e6f-4111-9298-95b11e8acfe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:41.559001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:41.559001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:41.559001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:41.559001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:41.559001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:41.559001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:41.559001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:41.559001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:41.559001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:41.559001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:41.559001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:41.559001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:41.559001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:41.559001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:41.559001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:41.559956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:41.559956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:41.559956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:41.559956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:41.559956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:41.559956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:41.559956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:41.559956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:41.559956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:41.559956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:41.559956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:41.559956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:41.559956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:41.559956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:41.559956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:41.559956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:41.559956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:41.560860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:41.560860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:41.560860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:41.560860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 284/565] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:41.561489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c220b36-d006-4d8f-a7ad-8e50338cba9b None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:41.561584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c220b36-d006-4d8f-a7ad-8e50338cba9b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:41.561815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c220b36-d006-4d8f-a7ad-8e50338cba9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:41.581515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-0c220b36-d006-4d8f-a7ad-8e50338cba9b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:41.582653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0c220b36-d006-4d8f-a7ad-8e50338cba9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:41.609447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46eeec4-6fda-4fc0-91de-d3140ad9161d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:41.642813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c250d-1083-42e6-99f0-2f391ae739f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:41.643449 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c250d-1083-42e6-99f0-2f391ae739f4 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:41.643733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c250d-1083-42e6-99f0-2f391ae739f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:41.644010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c250d-1083-42e6-99f0-2f391ae739f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:41.649918 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 285/566] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:41.650744 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c220b36-d006-4d8f-a7ad-8e50338cba9b None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:41.669681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:41.670757 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c250d-1083-42e6-99f0-2f391ae739f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:41.672261 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c250d-1083-42e6-99f0-2f391ae739f4 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:41.693319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-073c250d-1083-42e6-99f0-2f391ae739f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:41.693319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:41.693319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:41.693319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:41.693319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:41.693319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:41.693319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:41.693319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:41.693319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:41.693319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:41.693319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:41.693319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:41.693319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:41.693319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:41.693319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:41.693319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:41.694822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:41.694822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:41.694822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:41.694822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:41.694822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:41.694822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:41.694822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:41.694822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:41.694822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:41.694822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:41.694822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:41.694822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:41.694822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:41.694822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:41.694822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:41.694822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:41.694822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:41.695746 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:41.695746 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:41.695746 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:41.695746 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 282/567] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:41.696957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c250d-1083-42e6-99f0-2f391ae739f4 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:41.756048 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c220b36-d006-4d8f-a7ad-8e50338cba9b None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:41.792854 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8db6c-ca79-4f2f-a57a-81a8f2733254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:41.793449 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8db6c-ca79-4f2f-a57a-81a8f2733254 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:41.793770 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8db6c-ca79-4f2f-a57a-81a8f2733254 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:41.794000 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8db6c-ca79-4f2f-a57a-81a8f2733254 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:41.803681 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:41.813557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c250d-1083-42e6-99f0-2f391ae739f4 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:41.847790 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12a318-6675-4c49-bbac-645114fcc604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:41.848320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12a318-6675-4c49-bbac-645114fcc604 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:41.848599 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12a318-6675-4c49-bbac-645114fcc604 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:41.848937 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12a318-6675-4c49-bbac-645114fcc604 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:41.849557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12a318-6675-4c49-bbac-645114fcc604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:41.849863 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12a318-6675-4c49-bbac-645114fcc604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:41.851275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12a318-6675-4c49-bbac-645114fcc604 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:41.862767 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 283/568] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:41.863559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12a318-6675-4c49-bbac-645114fcc604 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:41.870409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e8db6c-ca79-4f2f-a57a-81a8f2733254 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:41.871953 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e8db6c-ca79-4f2f-a57a-81a8f2733254 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:41.928269 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:41.963770 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12a318-6675-4c49-bbac-645114fcc604 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:41.996137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d268b-e38b-4100-bb1f-bb0d33593179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:41.996764 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d268b-e38b-4100-bb1f-bb0d33593179 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:41.997038 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d268b-e38b-4100-bb1f-bb0d33593179 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:41.997310 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d268b-e38b-4100-bb1f-bb0d33593179 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:42.007279 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d268b-e38b-4100-bb1f-bb0d33593179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:42.008071 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 286/569] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:42.008767 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8db6c-ca79-4f2f-a57a-81a8f2733254 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:42.012958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-452d268b-e38b-4100-bb1f-bb0d33593179 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:42.012958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:42.012958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:42.012958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:42.012958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:42.012958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:42.012958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:42.012958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:42.012958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:42.012958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:42.012958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:42.012958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:42.012958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:42.012958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:42.012958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:42.012958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:42.013870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:42.013870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:42.013870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:42.013870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:42.013870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:42.013870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:42.013870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:42.013870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:42.013870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:42.013870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:42.013870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:42.013870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:42.013870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:42.013870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:42.013870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:42.013870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:42.015390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:42.015390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:42.015390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:42.015390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:42.015390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:42.015390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:42.015390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:42.015390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:42.015390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:42.015390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:42.015390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:42.015390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:42.015390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 284/570] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:42.015390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d268b-e38b-4100-bb1f-bb0d33593179 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:42.030557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:42.114773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8db6c-ca79-4f2f-a57a-81a8f2733254 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:42.122634 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d268b-e38b-4100-bb1f-bb0d33593179 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:42.132656 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:42.150099 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a533060-6516-40e5-93a3-cbd7af52aaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:42.150706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a533060-6516-40e5-93a3-cbd7af52aaaa None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:42.151013 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a533060-6516-40e5-93a3-cbd7af52aaaa None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:42.151288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a533060-6516-40e5-93a3-cbd7af52aaaa None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:42.152026 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a533060-6516-40e5-93a3-cbd7af52aaaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:42.152325 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a533060-6516-40e5-93a3-cbd7af52aaaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:42.153713 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a533060-6516-40e5-93a3-cbd7af52aaaa None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:42.155342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ec15a-05f5-44ff-bd44-26862405d97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:42.155963 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ec15a-05f5-44ff-bd44-26862405d97a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:42.156244 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ec15a-05f5-44ff-bd44-26862405d97a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:42.156517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ec15a-05f5-44ff-bd44-26862405d97a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:42.163738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ec15a-05f5-44ff-bd44-26862405d97a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:42.165143 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 287/571] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:42.165297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ec15a-05f5-44ff-bd44-26862405d97a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:42.166647 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a533060-6516-40e5-93a3-cbd7af52aaaa None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:42.176376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-9b5ec15a-05f5-44ff-bd44-26862405d97a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 05 14:14:42.176376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:42.176376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:42.176376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:42.176376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:42.176376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:42.176376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:42.176376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:42.176376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:42.176376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:42.176376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:42.176376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:42.176376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:42.176376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:42.176376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:42.176376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:42.177421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:42.178467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:42.178467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:42.178467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:42.178467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:42.178467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:42.178467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:42.178467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:42.178467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:42.178467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 05 14:14:42.178467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:42.178467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 285/572] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:42.178467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ec15a-05f5-44ff-bd44-26862405d97a None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:42.248675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:42.267527 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a533060-6516-40e5-93a3-cbd7af52aaaa None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:42.277761 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ec15a-05f5-44ff-bd44-26862405d97a None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:42.301443 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360b77a-4b19-4164-a9bf-bd7fb19d63c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:42.302064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360b77a-4b19-4164-a9bf-bd7fb19d63c5 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:42.302341 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360b77a-4b19-4164-a9bf-bd7fb19d63c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:42.302616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360b77a-4b19-4164-a9bf-bd7fb19d63c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:42.311979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc18d0b-b8fe-47ca-b237-9e5ad8ddf977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:42.312273 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3360b77a-4b19-4164-a9bf-bd7fb19d63c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:42.312595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc18d0b-b8fe-47ca-b237-9e5ad8ddf977 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:42.312876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc18d0b-b8fe-47ca-b237-9e5ad8ddf977 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:42.313148 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc18d0b-b8fe-47ca-b237-9e5ad8ddf977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:42.317610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-3360b77a-4b19-4164-a9bf-bd7fb19d63c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:42.317610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:42.317610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:42.317610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:42.317610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:42.317610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:42.317610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:42.317610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:42.317610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:42.317610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:42.317610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:42.317610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:42.317610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:42.317610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:42.317610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:42.317610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:42.318540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:42.318540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:42.318540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:42.318540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:42.318540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:42.318540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:42.318540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:42.318540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:42.318540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:42.318540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:42.318540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:42.318540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:42.318540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:42.318540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:42.318540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:42.318540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:42.319424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:42.319424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:42.319424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:42.319424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:42.319424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:42.319424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:42.319424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:42.319424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:42.319424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:42.319424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:42.319424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:42.319424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:42.319424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 288/573] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:42.320140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360b77a-4b19-4164-a9bf-bd7fb19d63c5 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:42.326685 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc18d0b-b8fe-47ca-b237-9e5ad8ddf977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:42.328181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc18d0b-b8fe-47ca-b237-9e5ad8ddf977 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:42.342029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-fdc18d0b-b8fe-47ca-b237-9e5ad8ddf977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:42.342029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:42.342029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:42.342029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:42.342029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:42.342029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:42.342029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:42.342029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:42.342029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:42.342029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:42.342029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:42.342029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:42.342029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:42.342029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:42.342029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:42.342029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:42.343182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:42.343182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:42.343182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:42.343182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:42.343182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:42.343182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:42.343182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:42.343182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:42.343182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:42.343182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:42.343182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:42.343182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:42.343182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:42.343182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:42.343182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:42.343182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:42.343182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:42.344266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:42.344266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:42.344266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:42.344266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 286/574] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:42.344266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc18d0b-b8fe-47ca-b237-9e5ad8ddf977 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:42.362892 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:42.421407 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360b77a-4b19-4164-a9bf-bd7fb19d63c5 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:42.443187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc18d0b-b8fe-47ca-b237-9e5ad8ddf977 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:42.453972 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b851ffa-a688-4cd0-8545-9e2cf5d847e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:42.454575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b851ffa-a688-4cd0-8545-9e2cf5d847e1 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:42.454885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b851ffa-a688-4cd0-8545-9e2cf5d847e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:42.455166 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b851ffa-a688-4cd0-8545-9e2cf5d847e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:42.455822 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b851ffa-a688-4cd0-8545-9e2cf5d847e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:42.456129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b851ffa-a688-4cd0-8545-9e2cf5d847e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:42.457498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b851ffa-a688-4cd0-8545-9e2cf5d847e1 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:42.464389 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:42.465146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 289/575] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:14:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:42.465902 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b851ffa-a688-4cd0-8545-9e2cf5d847e1 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:42.478278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9320e3-d218-43ff-a8f8-490c651459e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:42.478936 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9320e3-d218-43ff-a8f8-490c651459e2 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:42.479223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9320e3-d218-43ff-a8f8-490c651459e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:42.479494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9320e3-d218-43ff-a8f8-490c651459e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:42.486004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9320e3-d218-43ff-a8f8-490c651459e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:42.487494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9320e3-d218-43ff-a8f8-490c651459e2 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:42.498827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-6d9320e3-d218-43ff-a8f8-490c651459e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:42.498827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:42.498827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:42.498827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:42.498827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:42.498827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:42.498827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:42.498827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:42.498827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:42.498827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:42.498827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:42.498827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:42.498827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:14:42.498827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:14:42.498827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:14:42.499735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:14:42.499735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:42.499735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:42.499735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:42.499735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:42.499735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:42.499735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:42.499735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:42.499735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:42.499735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:42.499735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:42.499735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:42.499735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:42.499735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:42.499735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:42.499735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:42.499735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:42.500657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:14:42.500657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:14:42.500657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:14:42.500657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:14:42.500657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:14:42.500657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:14:42.500657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:42.500657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:42.500657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 287/576] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:42.501337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9320e3-d218-43ff-a8f8-490c651459e2 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:42.562232 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:42.565099 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b851ffa-a688-4cd0-8545-9e2cf5d847e1 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:42.597695 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b7514-de56-4478-9fd0-c2a98f8633ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:42.598266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9320e3-d218-43ff-a8f8-490c651459e2 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:42.598370 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b7514-de56-4478-9fd0-c2a98f8633ff None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:42.598553 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b7514-de56-4478-9fd0-c2a98f8633ff None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:42.598858 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b7514-de56-4478-9fd0-c2a98f8633ff None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:42.599578 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b7514-de56-4478-9fd0-c2a98f8633ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:42.599880 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b7514-de56-4478-9fd0-c2a98f8633ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:42.601283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b7514-de56-4478-9fd0-c2a98f8633ff None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:42.618810 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 290/577] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Sep 5 14:14:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:42.619616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b7514-de56-4478-9fd0-c2a98f8633ff None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:42.631771 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf18ff-7159-4b27-80c1-3b2c9449d0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:42.632374 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf18ff-7159-4b27-80c1-3b2c9449d0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:42.632654 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf18ff-7159-4b27-80c1-3b2c9449d0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:42.632928 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf18ff-7159-4b27-80c1-3b2c9449d0a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:42.633582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf18ff-7159-4b27-80c1-3b2c9449d0a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:42.633884 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf18ff-7159-4b27-80c1-3b2c9449d0a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:42.635299 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf18ff-7159-4b27-80c1-3b2c9449d0a0 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:42.643274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 288/578] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:14:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:42.644048 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf18ff-7159-4b27-80c1-3b2c9449d0a0 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:42.692842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:42.717290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b7514-de56-4478-9fd0-c2a98f8633ff None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:42.747252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf18ff-7159-4b27-80c1-3b2c9449d0a0 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:42.750311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca2a58-e814-4498-8a87-178a6e2f34a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:42.750946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca2a58-e814-4498-8a87-178a6e2f34a9 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:42.751228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca2a58-e814-4498-8a87-178a6e2f34a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:42.751495 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca2a58-e814-4498-8a87-178a6e2f34a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:42.752129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca2a58-e814-4498-8a87-178a6e2f34a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:42.752436 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca2a58-e814-4498-8a87-178a6e2f34a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:42.753825 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca2a58-e814-4498-8a87-178a6e2f34a9 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:42.765234 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 291/579] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:42.766005 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca2a58-e814-4498-8a87-178a6e2f34a9 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:42.783076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f657ed-43cf-437c-9192-743bca2f81cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:42.783697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f657ed-43cf-437c-9192-743bca2f81cf None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:42.783980 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f657ed-43cf-437c-9192-743bca2f81cf None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:42.784254 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f657ed-43cf-437c-9192-743bca2f81cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:42.785208 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:42.794608 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f657ed-43cf-437c-9192-743bca2f81cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:42.796140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f657ed-43cf-437c-9192-743bca2f81cf None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:42.810180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-b4f657ed-43cf-437c-9192-743bca2f81cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:42.810180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:42.810180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:42.810180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:42.810180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:42.810180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:42.810180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:42.810180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:42.810180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:42.810180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:42.810180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:42.810180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:42.810180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:42.810180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:42.810180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:42.810180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:42.811379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:42.811379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:42.811379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:42.811379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:42.811379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:42.811379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:42.811379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:42.811379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:42.811379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:42.811379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:42.811379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:42.811379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:42.811379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:42.811379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:42.811379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:42.811379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:42.811379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:42.812617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:42.812617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:42.812617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:42.812617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 289/580] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:42.812617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f657ed-43cf-437c-9192-743bca2f81cf None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:42.866428 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca2a58-e814-4498-8a87-178a6e2f34a9 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:42.900167 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9bcca-f762-41d4-924f-099523e145b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:42.900832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9bcca-f762-41d4-924f-099523e145b6 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:42.901112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9bcca-f762-41d4-924f-099523e145b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:42.901389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9bcca-f762-41d4-924f-099523e145b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:42.902074 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9bcca-f762-41d4-924f-099523e145b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:42.902382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9bcca-f762-41d4-924f-099523e145b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:42.903844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9bcca-f762-41d4-924f-099523e145b6 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:42.908701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:42.909865 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f657ed-43cf-437c-9192-743bca2f81cf None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:42.915824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 292/581] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:14:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:42.916688 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9bcca-f762-41d4-924f-099523e145b6 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:42.960084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7e261-0f6d-4b0f-84ef-cdb23b4d3ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:42.960631 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7e261-0f6d-4b0f-84ef-cdb23b4d3ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:42.960908 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7e261-0f6d-4b0f-84ef-cdb23b4d3ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:42.961177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7e261-0f6d-4b0f-84ef-cdb23b4d3ad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:42.971774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7e261-0f6d-4b0f-84ef-cdb23b4d3ad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:42.973408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7e261-0f6d-4b0f-84ef-cdb23b4d3ad2 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:42.988130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-cad7e261-0f6d-4b0f-84ef-cdb23b4d3ad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:42.988130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:42.988130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:42.988130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:42.988130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:42.988130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:42.988130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:42.988130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:42.988130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:42.988130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:42.988130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:42.988130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:42.988130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:42.988130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:42.988130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:42.988130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:42.989247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:42.989247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:42.989247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:42.989247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:42.989247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:42.989247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:42.989247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:42.989247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:42.989247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:42.989247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:42.989247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:42.989247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:42.989247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:42.989247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:42.989247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:42.989247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:42.989247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:42.990359 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:42.990359 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:42.990359 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:42.993822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 290/582] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:42.994669 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7e261-0f6d-4b0f-84ef-cdb23b4d3ad2 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:43.002192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:43.023787 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9bcca-f762-41d4-924f-099523e145b6 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:43.059202 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4f105-1bac-4c65-9f54-291864bfde56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:43.059827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4f105-1bac-4c65-9f54-291864bfde56 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:43.060104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4f105-1bac-4c65-9f54-291864bfde56 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:43.060374 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4f105-1bac-4c65-9f54-291864bfde56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:43.070454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4f105-1bac-4c65-9f54-291864bfde56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:43.072416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4f105-1bac-4c65-9f54-291864bfde56 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:43.089117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-53b4f105-1bac-4c65-9f54-291864bfde56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:43.089117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:43.089117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:43.089117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:43.089117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:43.089117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:43.089117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:43.089117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:43.089117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:43.089117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:43.089117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:43.089117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:43.089117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:43.089117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:43.089117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:43.089117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:43.094862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:43.094862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:43.094862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:43.094862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:43.094862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:43.094862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:43.094862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:43.094862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:43.094862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:43.094862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:43.094862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:43.094862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:43.094862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:43.094862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:43.094862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:43.094862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:43.094862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:43.096003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:43.096003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:43.096003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:43.096003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 293/583] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:43.096003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4f105-1bac-4c65-9f54-291864bfde56 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:43.096859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7e261-0f6d-4b0f-84ef-cdb23b4d3ad2 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:43.130656 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d613cf9-292e-4690-8351-be3c46e37ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:43.131312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d613cf9-292e-4690-8351-be3c46e37ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:43.131598 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d613cf9-292e-4690-8351-be3c46e37ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:43.131882 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d613cf9-292e-4690-8351-be3c46e37ea9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:43.138443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d613cf9-292e-4690-8351-be3c46e37ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:43.139941 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d613cf9-292e-4690-8351-be3c46e37ea9 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:43.151125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-7d613cf9-292e-4690-8351-be3c46e37ea9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:43.151125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:43.151125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:43.151125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:43.151125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:43.151125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:43.151125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:43.151125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:43.151125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:43.151125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:43.151125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:43.151125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:43.151125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:14:43.151125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:14:43.151125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:14:43.152138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:14:43.152138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:43.152138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:43.152138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:43.152138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:43.152138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:43.152138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:43.152138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:43.152138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:43.152138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:43.152138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:43.152138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:43.152138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:43.152138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:43.152138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:43.152138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:43.152138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:43.153223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:14:43.153223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:14:43.153223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:14:43.153223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:14:43.153223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:14:43.153223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:14:43.153223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:43.153223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:43.156874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 291/584] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:43.157620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d613cf9-292e-4690-8351-be3c46e37ea9 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:43.168504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:43.191827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4f105-1bac-4c65-9f54-291864bfde56 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:43.220437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:43.225910 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b7537-1731-487b-bf8d-197c333c2c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:43.226549 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b7537-1731-487b-bf8d-197c333c2c03 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:43.226856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b7537-1731-487b-bf8d-197c333c2c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:43.235041 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b7537-1731-487b-bf8d-197c333c2c03 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:43.235041 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b7537-1731-487b-bf8d-197c333c2c03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:43.235041 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b7537-1731-487b-bf8d-197c333c2c03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:43.235041 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b7537-1731-487b-bf8d-197c333c2c03 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:43.241180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 294/585] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:43.241956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b7537-1731-487b-bf8d-197c333c2c03 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:43.256954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d613cf9-292e-4690-8351-be3c46e37ea9 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:43.289573 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f881f11-21b9-4660-b477-f141305df113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:43.290742 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f881f11-21b9-4660-b477-f141305df113 None admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:43.291048 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f881f11-21b9-4660-b477-f141305df113 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:43.291321 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f881f11-21b9-4660-b477-f141305df113 None admin] PATH_INFO: `/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/709b9ae82dcd47408fc9a6d54a54428a/roles/0a83e40e2d64470a8489bf548e34dabd` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:43.304974 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f881f11-21b9-4660-b477-f141305df113 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9ca42deaed344eea726fe6e32356efc, user_id=709b9ae82dcd47408fc9a6d54a54428a, role_id=0a83e40e2d64470a8489bf548e34dabd)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:43.307097 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f881f11-21b9-4660-b477-f141305df113 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:43.340539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b7537-1731-487b-bf8d-197c333c2c03 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:43.375325 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ed500-6275-4680-8c21-9cf4c6bed72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:43.375926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ed500-6275-4680-8c21-9cf4c6bed72b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:43.376208 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ed500-6275-4680-8c21-9cf4c6bed72b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:43.376481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ed500-6275-4680-8c21-9cf4c6bed72b None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:43.377116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ed500-6275-4680-8c21-9cf4c6bed72b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:43.377418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ed500-6275-4680-8c21-9cf4c6bed72b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:43.378806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ed500-6275-4680-8c21-9cf4c6bed72b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:43.389934 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 295/586] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:43.390738 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ed500-6275-4680-8c21-9cf4c6bed72b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:43.393041 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 292/587] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:14:43 2022] PUT /identity/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/709b9ae82dcd47408fc9a6d54a54428a/roles/0a83e40e2d64470a8489bf548e34dabd => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:14:43.394015 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f881f11-21b9-4660-b477-f141305df113 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:43.455449 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:43.474548 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:43.513439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ed500-6275-4680-8c21-9cf4c6bed72b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:43.520125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f881f11-21b9-4660-b477-f141305df113 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:43.553891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6aee4-8802-430b-8bda-13e01f8edc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:43.554509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6aee4-8802-430b-8bda-13e01f8edc4d None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:43.554805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6aee4-8802-430b-8bda-13e01f8edc4d None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:43.555092 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6aee4-8802-430b-8bda-13e01f8edc4d None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:43.555736 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb6aee4-8802-430b-8bda-13e01f8edc4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:43.556570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb6aee4-8802-430b-8bda-13e01f8edc4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:43.557959 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb6aee4-8802-430b-8bda-13e01f8edc4d None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:43.563733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be307022-7b32-44b8-9c0d-c16d91f8c80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:43.564351 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be307022-7b32-44b8-9c0d-c16d91f8c80e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:43.564635 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be307022-7b32-44b8-9c0d-c16d91f8c80e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:43.564911 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be307022-7b32-44b8-9c0d-c16d91f8c80e None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:43.565918 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be307022-7b32-44b8-9c0d-c16d91f8c80e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:43.566225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be307022-7b32-44b8-9c0d-c16d91f8c80e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:43.567698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be307022-7b32-44b8-9c0d-c16d91f8c80e None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:43.570488 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 296/588] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:43.571351 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6aee4-8802-430b-8bda-13e01f8edc4d None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:43.580843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 293/589] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:14:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:43.581721 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be307022-7b32-44b8-9c0d-c16d91f8c80e None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:43.675478 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6aee4-8802-430b-8bda-13e01f8edc4d None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:43.687956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:43.692164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be307022-7b32-44b8-9c0d-c16d91f8c80e None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:43.711432 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:43.714340 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5f99c-d484-4420-946e-e46c4cb8e9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:43.715440 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5f99c-d484-4420-946e-e46c4cb8e9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:43.715729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5f99c-d484-4420-946e-e46c4cb8e9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:43.716167 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5f99c-d484-4420-946e-e46c4cb8e9d7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:43.723543 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5f99c-d484-4420-946e-e46c4cb8e9d7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:43.725654 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5f99c-d484-4420-946e-e46c4cb8e9d7 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:43.730742 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c144a-5146-445e-bbab-efbde46cec6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:43.734943 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c144a-5146-445e-bbab-efbde46cec6f None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:43.734943 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c144a-5146-445e-bbab-efbde46cec6f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:43.734943 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c144a-5146-445e-bbab-efbde46cec6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:43.738104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-f1c5f99c-d484-4420-946e-e46c4cb8e9d7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 14:14:43.738104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:43.738104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:43.738104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:43.738104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:43.738104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:43.738104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:43.738104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:43.738104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:43.738104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:43.738104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:43.738104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:43.738104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:43.738104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:43.738104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:43.738104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:43.739070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:43.740090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:43.740090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:43.740090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:43.740090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:43.740090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:43.740090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:43.740090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:43.740090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:43.740090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 14:14:43.740090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:43.740090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 297/590] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:43.740090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5f99c-d484-4420-946e-e46c4cb8e9d7 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:43.743165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c144a-5146-445e-bbab-efbde46cec6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:43.745638 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c144a-5146-445e-bbab-efbde46cec6f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:43.758630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-746c144a-5146-445e-bbab-efbde46cec6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:43.758630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:43.758630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:43.758630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:43.758630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:43.758630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:43.758630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:43.758630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:43.758630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:43.758630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:43.758630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:43.758630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:43.758630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:14:43.758630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:14:43.758630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:14:43.760301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:14:43.760301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:43.760301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:43.760301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:43.760301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:43.760301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:43.760301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:43.760301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:43.760301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:43.760301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:43.760301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:43.760301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:43.760301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:43.760301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:43.760301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:43.760301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:43.760301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:43.761314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:14:43.761314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:14:43.761314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:14:43.761314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:14:43.761314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:14:43.761314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:14:43.761314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:43.761314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:43.761314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 294/591] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:43.761843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c144a-5146-445e-bbab-efbde46cec6f None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:43.856226 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5f99c-d484-4420-946e-e46c4cb8e9d7 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:43.870166 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c144a-5146-445e-bbab-efbde46cec6f None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:43.895681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903c852-6b34-409b-8d7d-ecb45c6445b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:43.896333 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903c852-6b34-409b-8d7d-ecb45c6445b0 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:43.896939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903c852-6b34-409b-8d7d-ecb45c6445b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:43.897220 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903c852-6b34-409b-8d7d-ecb45c6445b0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:43.905232 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903c852-6b34-409b-8d7d-ecb45c6445b0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:43.906719 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903c852-6b34-409b-8d7d-ecb45c6445b0 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:43.907622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ad4a9-d0b2-4587-9bf4-2ec2cbafd6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:43.908236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ad4a9-d0b2-4587-9bf4-2ec2cbafd6fc None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:43.908532 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ad4a9-d0b2-4587-9bf4-2ec2cbafd6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:43.908796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ad4a9-d0b2-4587-9bf4-2ec2cbafd6fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:43.909599 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ad4a9-d0b2-4587-9bf4-2ec2cbafd6fc None admin] RBAC: Adding query filter params (role.id=0a83e40e2d64470a8489bf548e34dabd, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=0a85809d14cb449d848f22a3f39bf641) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:43.909834 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ad4a9-d0b2-4587-9bf4-2ec2cbafd6fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:43.911307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ad4a9-d0b2-4587-9bf4-2ec2cbafd6fc None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:43.927144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 295/592] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:14:43 2022] GET /identity/v3/role_assignments?user.id=0a85809d14cb449d848f22a3f39bf641&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=0a83e40e2d64470a8489bf548e34dabd => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:43.927144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ad4a9-d0b2-4587-9bf4-2ec2cbafd6fc None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:43.927408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-c903c852-6b34-409b-8d7d-ecb45c6445b0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 14:14:43.927408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:43.927408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:43.927408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:43.927408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:43.927408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:43.927408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:43.927408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:43.927408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:43.927408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:43.927408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:43.927408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:43.927408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:43.927408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:43.927408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:43.927408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:43.928330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:43.929291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:43.929291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:43.929291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:43.929291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:43.929291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:43.929291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:43.929291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:43.929291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:43.929291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 14:14:43.929291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:43.929291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 298/593] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:43.929291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903c852-6b34-409b-8d7d-ecb45c6445b0 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:43.977203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:43.992407 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:44.059239 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903c852-6b34-409b-8d7d-ecb45c6445b0 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:44.061582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ad4a9-d0b2-4587-9bf4-2ec2cbafd6fc None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:44.100623 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8d4d5-3436-436e-9472-5fc1acc94afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:44.101364 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8d4d5-3436-436e-9472-5fc1acc94afd None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:44.101742 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8d4d5-3436-436e-9472-5fc1acc94afd None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:44.102111 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8d4d5-3436-436e-9472-5fc1acc94afd None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:44.103024 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8d4d5-3436-436e-9472-5fc1acc94afd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:44.103434 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8d4d5-3436-436e-9472-5fc1acc94afd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:44.105018 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8d4d5-3436-436e-9472-5fc1acc94afd None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:44.105138 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336720d1-6926-466b-8e73-0d467669e13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:44.105744 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336720d1-6926-466b-8e73-0d467669e13b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:44.106026 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336720d1-6926-466b-8e73-0d467669e13b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:44.106304 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336720d1-6926-466b-8e73-0d467669e13b None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:44.107035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336720d1-6926-466b-8e73-0d467669e13b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:44.107349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336720d1-6926-466b-8e73-0d467669e13b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:44.108783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336720d1-6926-466b-8e73-0d467669e13b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:44.122809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 299/594] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:14:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:44.123748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336720d1-6926-466b-8e73-0d467669e13b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:44.129846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 296/595] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Sep 5 14:14:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:44.135495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8d4d5-3436-436e-9472-5fc1acc94afd None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:44.217472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:44.232500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336720d1-6926-466b-8e73-0d467669e13b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:44.245407 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:44.249087 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8d4d5-3436-436e-9472-5fc1acc94afd None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:44.265098 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff0ccd-1647-4af4-bf72-56e86be9a3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:44.265857 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff0ccd-1647-4af4-bf72-56e86be9a3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:44.271271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff0ccd-1647-4af4-bf72-56e86be9a3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:44.271271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff0ccd-1647-4af4-bf72-56e86be9a3e0 None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:44.271271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff0ccd-1647-4af4-bf72-56e86be9a3e0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:44.271271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff0ccd-1647-4af4-bf72-56e86be9a3e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:44.271271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff0ccd-1647-4af4-bf72-56e86be9a3e0 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:44.288414 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff13bdc-ea0c-4ca0-8341-6ca9f8781fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:44.290302 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff13bdc-ea0c-4ca0-8341-6ca9f8781fac None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:44.290725 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff13bdc-ea0c-4ca0-8341-6ca9f8781fac None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:44.291184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff13bdc-ea0c-4ca0-8341-6ca9f8781fac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:44.292116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 300/596] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Sep 5 14:14:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:44.293058 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff0ccd-1647-4af4-bf72-56e86be9a3e0 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:44.303381 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff13bdc-ea0c-4ca0-8341-6ca9f8781fac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:44.305290 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff13bdc-ea0c-4ca0-8341-6ca9f8781fac None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:44.322921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-8ff13bdc-ea0c-4ca0-8341-6ca9f8781fac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:44.322921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:44.322921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:44.322921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:44.322921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:44.322921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:44.322921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:44.322921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:44.322921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:44.322921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:44.322921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:44.322921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:44.322921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:44.322921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:44.322921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:44.322921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:44.324068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:44.324068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:44.324068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:44.324068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:44.324068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:44.324068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:44.324068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:44.324068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:44.324068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:44.324068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:44.324068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:44.324068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:44.324068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:44.324068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:44.324068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:44.324068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:44.324068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:44.325137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:44.325137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:44.325137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:44.331582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 297/597] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:14:44.332593 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff13bdc-ea0c-4ca0-8341-6ca9f8781fac None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:44.404248 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff0ccd-1647-4af4-bf72-56e86be9a3e0 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:44.447301 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8fe2b-7355-4a3d-a069-ef89bd1927e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:44.448129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8fe2b-7355-4a3d-a069-ef89bd1927e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:44.448649 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8fe2b-7355-4a3d-a069-ef89bd1927e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:44.449025 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8fe2b-7355-4a3d-a069-ef89bd1927e8 None admin] PATH_INFO: `/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/735986478c354ed196c601f12f3ac377/roles/0a83e40e2d64470a8489bf548e34dabd` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:44.455616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff13bdc-ea0c-4ca0-8341-6ca9f8781fac None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:44.465767 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8fe2b-7355-4a3d-a069-ef89bd1927e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9ca42deaed344eea726fe6e32356efc, user_id=735986478c354ed196c601f12f3ac377, role_id=0a83e40e2d64470a8489bf548e34dabd)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:44.467586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8fe2b-7355-4a3d-a069-ef89bd1927e8 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:44.495477 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b53b4-91b2-418f-9347-566905c8b493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:44.496696 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b53b4-91b2-418f-9347-566905c8b493 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:44.497144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b53b4-91b2-418f-9347-566905c8b493 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:44.497585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b53b4-91b2-418f-9347-566905c8b493 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:44.510226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b53b4-91b2-418f-9347-566905c8b493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:44.511999 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b53b4-91b2-418f-9347-566905c8b493 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:44.526547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-f62b53b4-91b2-418f-9347-566905c8b493 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:44.526547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:44.526547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:44.526547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:44.526547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:44.526547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:44.526547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:44.526547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:44.526547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:44.526547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:44.526547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:44.526547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:44.526547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:44.526547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:44.526547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:44.526547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:44.527714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:44.527714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:44.527714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:44.527714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:44.527714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:44.527714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:44.527714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:44.527714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:44.527714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:44.527714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:44.527714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:44.527714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:44.527714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:44.527714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:44.527714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:44.527714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:44.527714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:44.528804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:44.528804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:44.528804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:44.530471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 298/598] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:44.531403 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b53b4-91b2-418f-9347-566905c8b493 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:44.537736 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:44.541046 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:44.546220 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 301/599] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:14:44 2022] PUT /identity/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/735986478c354ed196c601f12f3ac377/roles/0a83e40e2d64470a8489bf548e34dabd => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:14:44.547862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8fe2b-7355-4a3d-a069-ef89bd1927e8 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:44.677079 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8fe2b-7355-4a3d-a069-ef89bd1927e8 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:44.720016 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f959e36-9df5-45b8-aa27-3c4894e5bc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:44.721593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f959e36-9df5-45b8-aa27-3c4894e5bc58 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:44.722088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f959e36-9df5-45b8-aa27-3c4894e5bc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:44.723166 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f959e36-9df5-45b8-aa27-3c4894e5bc58 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:44.724050 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f959e36-9df5-45b8-aa27-3c4894e5bc58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:44.724629 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f959e36-9df5-45b8-aa27-3c4894e5bc58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:44.726348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f959e36-9df5-45b8-aa27-3c4894e5bc58 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:44.748357 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 302/600] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:44.749962 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f959e36-9df5-45b8-aa27-3c4894e5bc58 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:44.767742 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b53b4-91b2-418f-9347-566905c8b493 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:44.812869 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:44.819685 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647953cf-4a50-46c5-b536-4d1767cb227a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:44.820306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647953cf-4a50-46c5-b536-4d1767cb227a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:44.820586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647953cf-4a50-46c5-b536-4d1767cb227a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:44.820858 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647953cf-4a50-46c5-b536-4d1767cb227a None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:44.822176 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647953cf-4a50-46c5-b536-4d1767cb227a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:44.822574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647953cf-4a50-46c5-b536-4d1767cb227a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:44.824073 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647953cf-4a50-46c5-b536-4d1767cb227a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:44.839298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 299/601] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:44.846060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647953cf-4a50-46c5-b536-4d1767cb227a None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:44.862398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f959e36-9df5-45b8-aa27-3c4894e5bc58 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:44.899155 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208c773-4962-4c5b-8605-778a6a98ab86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:44.900710 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208c773-4962-4c5b-8605-778a6a98ab86 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:44.901437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208c773-4962-4c5b-8605-778a6a98ab86 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:44.902135 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208c773-4962-4c5b-8605-778a6a98ab86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:44.909818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208c773-4962-4c5b-8605-778a6a98ab86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:44.912518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208c773-4962-4c5b-8605-778a6a98ab86 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:44.916479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:44.926201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-d208c773-4962-4c5b-8605-778a6a98ab86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:44.926201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:44.926201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:44.926201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:44.926201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:44.926201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:44.926201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:44.926201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:44.926201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:44.926201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:44.926201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:44.926201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:44.926201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:14:44.926201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:14:44.926201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:14:44.927187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:14:44.927187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:44.927187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:44.927187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:44.927187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:44.927187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:44.927187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:44.927187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:44.927187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:44.927187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:44.927187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:44.927187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:44.927187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:44.927187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:44.927187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:44.927187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:44.927187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:44.928113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:14:44.928113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:14:44.928113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:14:44.928113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:14:44.928113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:14:44.928113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:14:44.928113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:44.928113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:44.932040 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 303/602] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:14:44.936977 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208c773-4962-4c5b-8605-778a6a98ab86 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:44.962214 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647953cf-4a50-46c5-b536-4d1767cb227a None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:45.000219 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8d88e-64c1-4612-b1f1-bbbd1d8755a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:45.000777 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8d88e-64c1-4612-b1f1-bbbd1d8755a0 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:45.001099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8d88e-64c1-4612-b1f1-bbbd1d8755a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:45.001381 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8d88e-64c1-4612-b1f1-bbbd1d8755a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:45.009912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8d88e-64c1-4612-b1f1-bbbd1d8755a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:45.011420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8d88e-64c1-4612-b1f1-bbbd1d8755a0 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:45.024242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-f2e8d88e-64c1-4612-b1f1-bbbd1d8755a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:45.024242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:45.024242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:45.024242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:45.024242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:45.024242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:45.024242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:45.024242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:45.024242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:45.024242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:45.024242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:45.024242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:45.024242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:14:45.024242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:14:45.024242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:14:45.025225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:14:45.025225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:45.025225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:45.025225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:45.025225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:45.025225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:45.025225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:45.025225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:45.025225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:45.025225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:45.025225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:45.025225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:45.025225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:45.025225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:45.025225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:45.025225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:45.025225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:45.026536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:14:45.026536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:14:45.026536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:14:45.026536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:14:45.026536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:14:45.026536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:14:45.026536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:45.026536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:45.026536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 300/603] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:45.031525 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8d88e-64c1-4612-b1f1-bbbd1d8755a0 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:45.053506 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208c773-4962-4c5b-8605-778a6a98ab86 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:45.072599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:45.095742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71dbe6-529e-4fa2-9000-204d8c72b919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:45.096641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71dbe6-529e-4fa2-9000-204d8c72b919 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:45.097157 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71dbe6-529e-4fa2-9000-204d8c72b919 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:45.097634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71dbe6-529e-4fa2-9000-204d8c72b919 None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:45.098565 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71dbe6-529e-4fa2-9000-204d8c72b919 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:45.099122 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71dbe6-529e-4fa2-9000-204d8c72b919 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:45.100876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71dbe6-529e-4fa2-9000-204d8c72b919 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:45.119161 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 304/604] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:14:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:45.121271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71dbe6-529e-4fa2-9000-204d8c72b919 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:45.146815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8d88e-64c1-4612-b1f1-bbbd1d8755a0 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:45.173333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:45.186303 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ed40d-723e-47bc-8f1f-dd44ec5033b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:45.187005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ed40d-723e-47bc-8f1f-dd44ec5033b8 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:45.187331 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ed40d-723e-47bc-8f1f-dd44ec5033b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:45.187608 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ed40d-723e-47bc-8f1f-dd44ec5033b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:45.188405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ed40d-723e-47bc-8f1f-dd44ec5033b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:45.188717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ed40d-723e-47bc-8f1f-dd44ec5033b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:45.190155 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ed40d-723e-47bc-8f1f-dd44ec5033b8 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:45.204741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 301/605] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:14:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:45.210464 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ed40d-723e-47bc-8f1f-dd44ec5033b8 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:45.229379 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71dbe6-529e-4fa2-9000-204d8c72b919 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:45.265049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b010c-25ab-46e2-b261-64c2f8b17534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:45.266043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b010c-25ab-46e2-b261-64c2f8b17534 None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:45.266564 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b010c-25ab-46e2-b261-64c2f8b17534 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:45.267060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b010c-25ab-46e2-b261-64c2f8b17534 None admin] PATH_INFO: `/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/5c73ca2785334fc496b5227d3be655c7/roles/0a83e40e2d64470a8489bf548e34dabd` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:45.282242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b010c-25ab-46e2-b261-64c2f8b17534 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9ca42deaed344eea726fe6e32356efc, user_id=5c73ca2785334fc496b5227d3be655c7, role_id=0a83e40e2d64470a8489bf548e34dabd)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:45.284137 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b010c-25ab-46e2-b261-64c2f8b17534 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:45.320670 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ed40d-723e-47bc-8f1f-dd44ec5033b8 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:45.357370 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511eaacb-6748-4af9-9cdd-799a0b759aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:45.358032 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511eaacb-6748-4af9-9cdd-799a0b759aae None admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:45.358316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511eaacb-6748-4af9-9cdd-799a0b759aae None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:45.358618 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511eaacb-6748-4af9-9cdd-799a0b759aae None admin] PATH_INFO: `/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/e49ac59302cb42f7942954d3859d3675/roles/0a83e40e2d64470a8489bf548e34dabd` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:45.374217 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511eaacb-6748-4af9-9cdd-799a0b759aae None admin] RBAC: Authorizing `identity:create_grant(project_id=d9ca42deaed344eea726fe6e32356efc, user_id=e49ac59302cb42f7942954d3859d3675, role_id=0a83e40e2d64470a8489bf548e34dabd)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:45.375754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511eaacb-6748-4af9-9cdd-799a0b759aae None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:45.385414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 305/606] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:14:45 2022] PUT /identity/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/5c73ca2785334fc496b5227d3be655c7/roles/0a83e40e2d64470a8489bf548e34dabd => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:14:45.447825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 302/607] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:14:45 2022] PUT /identity/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/e49ac59302cb42f7942954d3859d3675/roles/0a83e40e2d64470a8489bf548e34dabd => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:14:45.448611 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda44739-8a42-48a9-9cbd-4fe4c25a1fe8 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:45.448898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda44739-8a42-48a9-9cbd-4fe4c25a1fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:45.449173 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda44739-8a42-48a9-9cbd-4fe4c25a1fe8 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:45.450526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 306/608] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:14:45.472587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8a3cf-1a2a-4089-b7a6-4b62652d0b3c None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:45.472963 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8a3cf-1a2a-4089-b7a6-4b62652d0b3c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:45.478467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8a3cf-1a2a-4089-b7a6-4b62652d0b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:45.500033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-dfe8a3cf-1a2a-4089-b7a6-4b62652d0b3c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:45.501485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-dfe8a3cf-1a2a-4089-b7a6-4b62652d0b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:45.548298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:45.600837 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 303/609] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:45.881765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda44739-8a42-48a9-9cbd-4fe4c25a1fe8 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:45.918483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:46.020483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda44739-8a42-48a9-9cbd-4fe4c25a1fe8 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:46.055454 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906599f-c804-4073-bc78-a6a906f54732 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:46.056469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906599f-c804-4073-bc78-a6a906f54732 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:46.056853 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906599f-c804-4073-bc78-a6a906f54732 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:46.058535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 304/610] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:46.063452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7942e-883f-4e06-b7f6-8f35a69d2c35 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:46.063844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7942e-883f-4e06-b7f6-8f35a69d2c35 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:46.064250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7942e-883f-4e06-b7f6-8f35a69d2c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:46.066601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72353f16-8c4b-488a-84e5-1ae2f93d94cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:46.067551 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72353f16-8c4b-488a-84e5-1ae2f93d94cd None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:46.068040 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72353f16-8c4b-488a-84e5-1ae2f93d94cd None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:46.068560 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72353f16-8c4b-488a-84e5-1ae2f93d94cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:46.092251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72353f16-8c4b-488a-84e5-1ae2f93d94cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:46.098117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-72353f16-8c4b-488a-84e5-1ae2f93d94cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:46.098117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:46.098117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:46.098117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:46.098117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:46.098117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:46.098117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:46.098117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:46.098117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:46.098117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:46.098117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:46.098117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:46.098117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:46.098117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:46.098117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:46.098117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:46.099091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:46.099091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:46.099091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:46.099091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:46.099091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:46.099091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:46.099091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:46.099091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:46.099091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:46.099091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:46.099091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:46.099091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:46.099091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:46.099091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:46.099091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:46.099091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:46.099976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:46.099976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:46.099976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:46.099976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:46.099976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:46.099976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:46.099976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:46.099976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:46.099976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:46.099976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:46.099976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:46.099976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:46.102212 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 307/611] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:46.103494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-4cb7942e-883f-4e06-b7f6-8f35a69d2c35 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:46.104918 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-4cb7942e-883f-4e06-b7f6-8f35a69d2c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:46.107564 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72353f16-8c4b-488a-84e5-1ae2f93d94cd None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:46.234174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 305/612] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 14:14:46.258086 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:46.271577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72353f16-8c4b-488a-84e5-1ae2f93d94cd None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:46.308242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93138b89-a154-4ceb-ba94-0ddba520f380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:46.308851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93138b89-a154-4ceb-ba94-0ddba520f380 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:46.309132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93138b89-a154-4ceb-ba94-0ddba520f380 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:46.309412 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93138b89-a154-4ceb-ba94-0ddba520f380 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:46.310068 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93138b89-a154-4ceb-ba94-0ddba520f380 None admin] RBAC: Adding query filter params (name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:46.310372 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93138b89-a154-4ceb-ba94-0ddba520f380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:46.311848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93138b89-a154-4ceb-ba94-0ddba520f380 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:46.320787 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 308/613] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:14:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:46.326874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7942e-883f-4e06-b7f6-8f35a69d2c35 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:46.449321 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:46.455956 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7942e-883f-4e06-b7f6-8f35a69d2c35 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:46.491906 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb85a26-7bba-4f8f-8a00-2b0dfda9200f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:46.492516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb85a26-7bba-4f8f-8a00-2b0dfda9200f None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:46.492802 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb85a26-7bba-4f8f-8a00-2b0dfda9200f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:46.493130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb85a26-7bba-4f8f-8a00-2b0dfda9200f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:46.504017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb85a26-7bba-4f8f-8a00-2b0dfda9200f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:46.505467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb85a26-7bba-4f8f-8a00-2b0dfda9200f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:46.520677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-dfb85a26-7bba-4f8f-8a00-2b0dfda9200f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:46.520677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:46.520677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:46.520677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:46.520677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:46.520677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:46.520677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:46.520677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:46.520677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:46.520677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:46.520677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:46.520677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:46.520677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:46.520677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:46.520677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:46.520677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:46.521813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:46.521813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:46.521813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:46.521813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:46.521813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:46.521813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:46.521813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:46.521813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:46.521813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:46.521813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:46.521813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:46.521813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:46.521813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:46.521813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:46.521813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:46.521813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:46.521813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:46.523077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:46.523077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:46.523077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:46.523077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 306/614] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:14:46.527765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93138b89-a154-4ceb-ba94-0ddba520f380 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:46.624937 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb85a26-7bba-4f8f-8a00-2b0dfda9200f None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:46.635144 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93138b89-a154-4ceb-ba94-0ddba520f380 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:46.670284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b99e8-d099-4458-a78e-83af7fe41d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:46.670964 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b99e8-d099-4458-a78e-83af7fe41d06 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:46.671252 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b99e8-d099-4458-a78e-83af7fe41d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:46.671585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b99e8-d099-4458-a78e-83af7fe41d06 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:46.672231 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b99e8-d099-4458-a78e-83af7fe41d06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:46.672534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b99e8-d099-4458-a78e-83af7fe41d06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:46.673997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b99e8-d099-4458-a78e-83af7fe41d06 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:46.687269 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 309/615] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:46.693055 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b99e8-d099-4458-a78e-83af7fe41d06 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:46.705843 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:46.803602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b99e8-d099-4458-a78e-83af7fe41d06 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:46.846638 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829c227-1238-45ad-a9e3-1cdf05af2e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:46.847362 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829c227-1238-45ad-a9e3-1cdf05af2e5e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:46.847648 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829c227-1238-45ad-a9e3-1cdf05af2e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:46.847923 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829c227-1238-45ad-a9e3-1cdf05af2e5e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:46.856606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829c227-1238-45ad-a9e3-1cdf05af2e5e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:46.858173 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829c227-1238-45ad-a9e3-1cdf05af2e5e None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:46.869698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb85a26-7bba-4f8f-8a00-2b0dfda9200f None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:46.882437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-f829c227-1238-45ad-a9e3-1cdf05af2e5e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:14:46.882437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:46.882437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:46.882437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:46.882437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:46.882437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:46.882437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:46.882437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:46.882437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:46.882437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:46.882437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:46.882437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:46.882437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:46.882437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:46.882437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:46.882437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:46.883445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:46.884463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:46.884463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:46.884463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:46.884463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:46.884463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:46.884463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:46.884463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:46.884463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:46.884463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:14:46.884463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:46.890897 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 310/616] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 5 14:14:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:46.907016 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829c227-1238-45ad-a9e3-1cdf05af2e5e None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:46.934813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d2cde-dccb-4065-81fe-9d205ced7883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:46.934813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d2cde-dccb-4065-81fe-9d205ced7883 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:46.934813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d2cde-dccb-4065-81fe-9d205ced7883 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:46.934813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d2cde-dccb-4065-81fe-9d205ced7883 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:46.946662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d2cde-dccb-4065-81fe-9d205ced7883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:46.952492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-6f1d2cde-dccb-4065-81fe-9d205ced7883 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:46.952492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:46.952492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:46.952492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:46.952492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:46.952492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:46.952492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:46.952492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:46.952492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:46.952492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:46.952492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:46.952492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:46.952492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:46.952492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:46.952492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:46.952492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:46.953616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:46.953616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:46.953616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:46.953616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:46.953616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:46.953616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:46.953616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:46.953616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:46.953616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:46.953616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:46.953616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:46.953616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:46.953616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:46.953616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:46.953616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:46.953616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:46.954582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:46.954582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:46.954582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:46.954582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:46.954582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:46.954582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:46.954582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:46.954582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:46.954582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:46.954582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:46.954582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:46.954582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:46.956671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 307/617] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:14:46.962160 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d2cde-dccb-4065-81fe-9d205ced7883 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:47.019164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:47.077977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d2cde-dccb-4065-81fe-9d205ced7883 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:47.117182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87ab81-8faa-44e7-9bfc-7c766c3f5d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:47.117806 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87ab81-8faa-44e7-9bfc-7c766c3f5d7c None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:47.118093 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87ab81-8faa-44e7-9bfc-7c766c3f5d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:47.118967 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87ab81-8faa-44e7-9bfc-7c766c3f5d7c None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:47.119649 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87ab81-8faa-44e7-9bfc-7c766c3f5d7c None admin] RBAC: Adding query filter params (name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:47.119960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87ab81-8faa-44e7-9bfc-7c766c3f5d7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:47.121350 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87ab81-8faa-44e7-9bfc-7c766c3f5d7c None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:47.128303 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:47.142447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 308/618] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:14:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:47.151808 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87ab81-8faa-44e7-9bfc-7c766c3f5d7c None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:47.169165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829c227-1238-45ad-a9e3-1cdf05af2e5e None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:47.218358 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917fec9-9b68-4ee7-8f03-14d2bc9c110b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:47.219372 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917fec9-9b68-4ee7-8f03-14d2bc9c110b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:47.219868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917fec9-9b68-4ee7-8f03-14d2bc9c110b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:47.220367 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917fec9-9b68-4ee7-8f03-14d2bc9c110b None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:47.221301 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917fec9-9b68-4ee7-8f03-14d2bc9c110b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:47.221859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917fec9-9b68-4ee7-8f03-14d2bc9c110b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:47.223722 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917fec9-9b68-4ee7-8f03-14d2bc9c110b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:47.249827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 311/619] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 5 14:14:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:47.249827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0fa14-5ca6-4f46-856f-84c6aca05334 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:47.249827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0fa14-5ca6-4f46-856f-84c6aca05334 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:47.249827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0fa14-5ca6-4f46-856f-84c6aca05334 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:47.249827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 312/620] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:47.249827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0fa14-5ca6-4f46-856f-84c6aca05334 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:47.271206 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:47.312463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87ab81-8faa-44e7-9bfc-7c766c3f5d7c None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:47.359100 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e98b3-2ca1-4183-9bac-902e81c5c70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:47.360240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e98b3-2ca1-4183-9bac-902e81c5c70f None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:47.360674 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e98b3-2ca1-4183-9bac-902e81c5c70f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:47.361495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e98b3-2ca1-4183-9bac-902e81c5c70f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:47.365161 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0fa14-5ca6-4f46-856f-84c6aca05334 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:47.377300 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e98b3-2ca1-4183-9bac-902e81c5c70f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:47.379487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e98b3-2ca1-4183-9bac-902e81c5c70f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:47.397098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-277e98b3-2ca1-4183-9bac-902e81c5c70f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:47.397098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:47.397098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:47.397098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:47.397098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:47.397098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:47.397098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:47.397098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:47.397098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:47.397098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:47.397098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:47.397098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:47.397098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:47.397098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:47.397098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:47.397098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:47.398269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:47.398269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:47.398269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:47.398269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:47.398269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:47.398269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:47.398269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:47.398269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:47.398269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:47.398269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:47.398269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:47.398269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:47.398269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:47.398269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:47.398269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:47.398269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:47.398269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:47.399386 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:47.399386 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:47.399386 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:47.400465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:47.402242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 309/621] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:47.403821 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b7db5-f551-4794-8bb5-fe054e7d9f81 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:47.404318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b7db5-f551-4794-8bb5-fe054e7d9f81 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:47.404812 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b7db5-f551-4794-8bb5-fe054e7d9f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:47.414336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664fc49-7805-427e-aa4e-ae3226e254a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:47.415967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664fc49-7805-427e-aa4e-ae3226e254a4 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:47.415967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664fc49-7805-427e-aa4e-ae3226e254a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:47.415967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664fc49-7805-427e-aa4e-ae3226e254a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:47.426623 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-904b7db5-f551-4794-8bb5-fe054e7d9f81 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:47.428185 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-904b7db5-f551-4794-8bb5-fe054e7d9f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:47.488734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664fc49-7805-427e-aa4e-ae3226e254a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:47.490226 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664fc49-7805-427e-aa4e-ae3226e254a4 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:47.501017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 310/622] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:47.502123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b7db5-f551-4794-8bb5-fe054e7d9f81 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:47.560695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:47.614728 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b7db5-f551-4794-8bb5-fe054e7d9f81 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:47.651550 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684800f-a03b-453a-bb6d-2c989100eff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:47.652684 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684800f-a03b-453a-bb6d-2c989100eff8 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:47.653197 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684800f-a03b-453a-bb6d-2c989100eff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:47.653679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684800f-a03b-453a-bb6d-2c989100eff8 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:47.654629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684800f-a03b-453a-bb6d-2c989100eff8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:47.657509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684800f-a03b-453a-bb6d-2c989100eff8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:47.659142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684800f-a03b-453a-bb6d-2c989100eff8 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:47.676559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 311/623] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:47.684187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684800f-a03b-453a-bb6d-2c989100eff8 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:47.753954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 313/624] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 14:14:47.761565 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664fc49-7805-427e-aa4e-ae3226e254a4 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:47.807205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:47.820557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684800f-a03b-453a-bb6d-2c989100eff8 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:47.832778 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:47.859481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d795df-78fe-4eac-9879-f29ddef5f4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:47.860335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d795df-78fe-4eac-9879-f29ddef5f4ae None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:47.860795 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d795df-78fe-4eac-9879-f29ddef5f4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:47.861214 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d795df-78fe-4eac-9879-f29ddef5f4ae None admin] PATH_INFO: `/v3/users/swift` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:47.869139 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d795df-78fe-4eac-9879-f29ddef5f4ae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:47.870816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d795df-78fe-4eac-9879-f29ddef5f4ae None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:47.880310 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664fc49-7805-427e-aa4e-ae3226e254a4 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:47.883305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-27d795df-78fe-4eac-9879-f29ddef5f4ae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 14:14:47.883305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:47.883305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:47.883305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:47.883305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:47.883305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:47.883305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:47.883305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:47.883305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:47.883305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:47.883305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:47.883305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:47.883305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:47.883305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:47.883305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:47.883305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:47.884460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:47.885651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:47.885651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:47.885651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:47.885651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:47.885651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:47.885651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:47.885651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:47.885651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:47.885651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 14:14:47.885651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:47.887880 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 312/625] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:14:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:47.889188 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835439f5-e6b8-4f7e-981d-b44e38d96732 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:47.889620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835439f5-e6b8-4f7e-981d-b44e38d96732 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:47.890057 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835439f5-e6b8-4f7e-981d-b44e38d96732 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:47.891716 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 313/626] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:47.892536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835439f5-e6b8-4f7e-981d-b44e38d96732 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:47.921522 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77898d0-e887-46cc-8822-c01b71d75bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:47.922159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77898d0-e887-46cc-8822-c01b71d75bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:47.922448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77898d0-e887-46cc-8822-c01b71d75bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:47.922722 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77898d0-e887-46cc-8822-c01b71d75bc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:47.932615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77898d0-e887-46cc-8822-c01b71d75bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:47.934058 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77898d0-e887-46cc-8822-c01b71d75bc8 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:47.948595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-f77898d0-e887-46cc-8822-c01b71d75bc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:47.948595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:47.948595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:47.948595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:47.948595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:47.948595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:47.948595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:47.948595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:47.948595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:47.948595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:47.948595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:47.948595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:47.948595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:47.948595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:47.948595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:47.948595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:47.949742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:47.949742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:47.949742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:47.949742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:47.949742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:47.949742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:47.949742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:47.949742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:47.949742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:47.949742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:47.949742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:47.949742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:47.949742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:47.949742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:47.949742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:47.949742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:47.949742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:47.950925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:47.950925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:47.950925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:47.950925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 314/627] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:47.951191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce2f0d-4475-4a7d-85ac-80109939057e None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:47.951477 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce2f0d-4475-4a7d-85ac-80109939057e None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:47.951754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce2f0d-4475-4a7d-85ac-80109939057e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:47.969611 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-e1ce2f0d-4475-4a7d-85ac-80109939057e None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:47.970818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e1ce2f0d-4475-4a7d-85ac-80109939057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:47.999101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835439f5-e6b8-4f7e-981d-b44e38d96732 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:48.036273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc5f5e5-bd04-4eb2-8cce-8f6acaedfe92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:48.036556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 315/628] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:48.036900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5f5e5-bd04-4eb2-8cce-8f6acaedfe92 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:48.037179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5f5e5-bd04-4eb2-8cce-8f6acaedfe92 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:48.037455 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5f5e5-bd04-4eb2-8cce-8f6acaedfe92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:48.037636 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce2f0d-4475-4a7d-85ac-80109939057e None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:48.047500 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc5f5e5-bd04-4eb2-8cce-8f6acaedfe92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:48.052274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-fbc5f5e5-bd04-4eb2-8cce-8f6acaedfe92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:48.052274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:48.052274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:48.052274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:48.052274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:48.052274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:48.052274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:48.052274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:48.052274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:48.052274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:48.052274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:48.052274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:48.052274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:48.052274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:48.052274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:48.052274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:48.053226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:48.053226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:48.053226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:48.053226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:48.053226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:48.053226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:48.053226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:48.053226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:48.053226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:48.053226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:48.053226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:48.053226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:48.053226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:48.053226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:48.053226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:48.053226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:48.055383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:48.055383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:48.055383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:48.055383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:48.055383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:48.055383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:48.055383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:48.055383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:48.055383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:48.055383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:48.055383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:48.055383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:48.055383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 314/629] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:48.055383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c03b9-8a37-499f-973b-3c0f4bf69aaf None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:48.061722 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c03b9-8a37-499f-973b-3c0f4bf69aaf None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:48.061722 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c03b9-8a37-499f-973b-3c0f4bf69aaf None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:48.061722 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 315/630] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:48.061722 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c03b9-8a37-499f-973b-3c0f4bf69aaf None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:48.065341 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:48.095813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:48.148575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce2f0d-4475-4a7d-85ac-80109939057e None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:48.172041 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c03b9-8a37-499f-973b-3c0f4bf69aaf None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:48.188762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eadefb-d439-4147-a235-40386598e518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:48.189380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eadefb-d439-4147-a235-40386598e518 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:48.189669 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eadefb-d439-4147-a235-40386598e518 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:48.189943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eadefb-d439-4147-a235-40386598e518 None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:48.190962 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eadefb-d439-4147-a235-40386598e518 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:48.191268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eadefb-d439-4147-a235-40386598e518 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:48.192705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eadefb-d439-4147-a235-40386598e518 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:48.208139 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba24d26-c63c-42f4-af57-bac85c4da9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:48.208719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba24d26-c63c-42f4-af57-bac85c4da9aa None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:48.209003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba24d26-c63c-42f4-af57-bac85c4da9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:48.209280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba24d26-c63c-42f4-af57-bac85c4da9aa None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:48.209976 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba24d26-c63c-42f4-af57-bac85c4da9aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:48.210224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba24d26-c63c-42f4-af57-bac85c4da9aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:48.211617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba24d26-c63c-42f4-af57-bac85c4da9aa None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:48.217076 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 316/631] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Sep 5 14:14:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:48.217076 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eadefb-d439-4147-a235-40386598e518 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:48.227806 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 316/632] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:48.229139 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427aca8-7e3b-43fa-a388-773cc7769a38 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:48.229498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427aca8-7e3b-43fa-a388-773cc7769a38 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:48.229844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427aca8-7e3b-43fa-a388-773cc7769a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:48.249775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-2427aca8-7e3b-43fa-a388-773cc7769a38 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:48.251081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-2427aca8-7e3b-43fa-a388-773cc7769a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:48.288635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:48.321070 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 317/633] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:48.322578 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427aca8-7e3b-43fa-a388-773cc7769a38 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:48.323828 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eadefb-d439-4147-a235-40386598e518 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:48.349482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:48.359231 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a669e69-2013-49f4-98e4-7dc140124436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:48.359919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a669e69-2013-49f4-98e4-7dc140124436 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:48.360277 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a669e69-2013-49f4-98e4-7dc140124436 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:48.360625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a669e69-2013-49f4-98e4-7dc140124436 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:48.361392 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a669e69-2013-49f4-98e4-7dc140124436 None admin] RBAC: Adding query filter params (name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:48.361821 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a669e69-2013-49f4-98e4-7dc140124436 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:48.363316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a669e69-2013-49f4-98e4-7dc140124436 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:48.372698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 317/634] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:14:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:48.373701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a669e69-2013-49f4-98e4-7dc140124436 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:48.433856 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427aca8-7e3b-43fa-a388-773cc7769a38 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:48.468680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9610af7-f8e4-419e-82bb-4aaada282e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:48.469318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9610af7-f8e4-419e-82bb-4aaada282e8d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:48.469603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9610af7-f8e4-419e-82bb-4aaada282e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:48.469880 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9610af7-f8e4-419e-82bb-4aaada282e8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:48.480548 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9610af7-f8e4-419e-82bb-4aaada282e8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:48.481724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a669e69-2013-49f4-98e4-7dc140124436 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:48.482002 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9610af7-f8e4-419e-82bb-4aaada282e8d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:48.498225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-c9610af7-f8e4-419e-82bb-4aaada282e8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:48.498225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:48.498225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:48.498225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:48.498225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:48.498225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:48.498225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:48.498225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:48.498225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:48.498225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:48.498225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:48.498225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:48.498225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:48.498225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:48.498225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:48.498225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:48.499390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:48.499390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:48.499390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:48.499390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:48.499390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:48.499390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:48.499390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:48.499390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:48.499390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:48.499390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:48.499390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:48.499390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:48.499390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:48.499390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:48.499390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:48.499390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:48.499390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:48.500446 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:48.500446 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:48.500446 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:48.503923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 318/635] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:48.504747 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9610af7-f8e4-419e-82bb-4aaada282e8d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:48.514342 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:48.520916 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b629a3-904d-4ade-9671-4a25f3f66d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:48.521612 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b629a3-904d-4ade-9671-4a25f3f66d06 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:48.522076 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b629a3-904d-4ade-9671-4a25f3f66d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:48.522451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b629a3-904d-4ade-9671-4a25f3f66d06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:48.529511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b629a3-904d-4ade-9671-4a25f3f66d06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:48.531138 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b629a3-904d-4ade-9671-4a25f3f66d06 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:48.543408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-24b629a3-904d-4ade-9671-4a25f3f66d06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:48.543408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:48.543408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:48.543408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:48.543408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:48.543408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:48.543408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:48.543408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:48.543408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:48.543408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:48.543408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:48.543408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:48.543408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:14:48.543408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:14:48.543408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:14:48.544514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:14:48.544514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:48.544514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:48.544514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:48.544514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:48.544514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:48.544514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:48.544514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:48.544514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:48.544514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:48.544514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:48.544514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:48.544514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:48.544514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:48.544514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:48.544514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:48.544514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:48.545848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:14:48.545848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:14:48.545848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:14:48.545848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:14:48.545848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:14:48.545848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:14:48.545848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:48.545848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:48.545848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 318/636] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:48.546513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b629a3-904d-4ade-9671-4a25f3f66d06 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:48.580842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:48.611221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9610af7-f8e4-419e-82bb-4aaada282e8d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:48.644138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f056ccf-bc78-47c1-a3d5-7c8cb77a021c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:48.644767 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f056ccf-bc78-47c1-a3d5-7c8cb77a021c None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:48.645058 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f056ccf-bc78-47c1-a3d5-7c8cb77a021c None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:48.645336 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f056ccf-bc78-47c1-a3d5-7c8cb77a021c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:48.648374 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b629a3-904d-4ade-9671-4a25f3f66d06 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:48.655943 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f056ccf-bc78-47c1-a3d5-7c8cb77a021c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:48.661184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-7f056ccf-bc78-47c1-a3d5-7c8cb77a021c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:48.661184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:48.661184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:48.661184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:48.661184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:48.661184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:48.661184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:48.661184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:48.661184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:48.661184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:48.661184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:48.661184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:48.661184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:48.661184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:48.661184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:48.661184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:48.662309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:48.662309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:48.662309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:48.662309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:48.662309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:48.662309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:48.662309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:48.662309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:48.662309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:48.662309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:48.662309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:48.662309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:48.662309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:48.662309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:48.662309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:48.662309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:48.663463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:48.663463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:48.663463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:48.663463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:48.663463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:48.663463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:48.663463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:48.663463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:48.663463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:48.663463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:48.663463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:48.663463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:48.663463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 319/637] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:48.663463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f056ccf-bc78-47c1-a3d5-7c8cb77a021c None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:48.684638 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57377c5-72f9-46de-a1cb-21deedcd7ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:48.685197 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57377c5-72f9-46de-a1cb-21deedcd7ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:48.685475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57377c5-72f9-46de-a1cb-21deedcd7ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:48.685740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57377c5-72f9-46de-a1cb-21deedcd7ac4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:48.696616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57377c5-72f9-46de-a1cb-21deedcd7ac4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:48.698061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57377c5-72f9-46de-a1cb-21deedcd7ac4 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:48.712626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-a57377c5-72f9-46de-a1cb-21deedcd7ac4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:48.712626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:48.712626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:48.712626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:48.712626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:48.712626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:48.712626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:48.712626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:48.712626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:48.712626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:48.712626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:48.712626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:48.712626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:48.712626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:48.712626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:48.712626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:48.713877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:48.713877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:48.713877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:48.713877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:48.713877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:48.713877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:48.713877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:48.713877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:48.713877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:48.713877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:48.713877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:48.713877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:48.713877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:48.713877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:48.713877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:48.713877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:48.713877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:48.714919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:48.714919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:48.714919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:48.714919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 319/638] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:48.714919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57377c5-72f9-46de-a1cb-21deedcd7ac4 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:48.756850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:48.764547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f056ccf-bc78-47c1-a3d5-7c8cb77a021c None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:48.793957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:48.798707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e70b0-fc54-45bd-b50c-bc9f32bca18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:48.799339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e70b0-fc54-45bd-b50c-bc9f32bca18d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:48.799621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e70b0-fc54-45bd-b50c-bc9f32bca18d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:48.799893 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e70b0-fc54-45bd-b50c-bc9f32bca18d None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:48.800522 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e70b0-fc54-45bd-b50c-bc9f32bca18d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:48.800834 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e70b0-fc54-45bd-b50c-bc9f32bca18d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:48.802199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e70b0-fc54-45bd-b50c-bc9f32bca18d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:48.812385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57377c5-72f9-46de-a1cb-21deedcd7ac4 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:48.813420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 320/639] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:48.814193 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e70b0-fc54-45bd-b50c-bc9f32bca18d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:48.852421 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79261771-16e6-4cbd-9c9c-257b02fa3b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:48.853141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79261771-16e6-4cbd-9c9c-257b02fa3b0a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:48.853427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79261771-16e6-4cbd-9c9c-257b02fa3b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:48.853705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79261771-16e6-4cbd-9c9c-257b02fa3b0a None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:48.854399 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79261771-16e6-4cbd-9c9c-257b02fa3b0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:48.854709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79261771-16e6-4cbd-9c9c-257b02fa3b0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:48.856171 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79261771-16e6-4cbd-9c9c-257b02fa3b0a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:48.867977 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 320/640] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:14:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:48.869297 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79261771-16e6-4cbd-9c9c-257b02fa3b0a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:48.917187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e70b0-fc54-45bd-b50c-bc9f32bca18d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:48.950646 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bd05c-f0b3-4eb5-be34-a17d8f63b170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:48.951238 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bd05c-f0b3-4eb5-be34-a17d8f63b170 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:48.951518 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bd05c-f0b3-4eb5-be34-a17d8f63b170 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:48.951794 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bd05c-f0b3-4eb5-be34-a17d8f63b170 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:48.961719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bd05c-f0b3-4eb5-be34-a17d8f63b170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:48.965487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79261771-16e6-4cbd-9c9c-257b02fa3b0a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:48.966982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-369bd05c-f0b3-4eb5-be34-a17d8f63b170 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:48.966982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:48.966982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:48.966982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:48.966982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:48.966982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:48.966982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:48.966982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:48.966982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:48.966982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:48.966982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:48.966982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:48.966982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:48.966982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:48.966982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:48.966982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:48.968184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:48.968184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:48.968184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:48.968184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:48.968184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:48.968184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:48.968184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:48.968184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:48.968184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:48.968184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:48.968184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:48.968184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:48.968184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:48.968184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:48.968184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:48.968184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:48.969264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:48.969264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:48.969264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:48.969264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:48.969264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:48.969264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:48.969264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:48.969264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:48.969264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:48.969264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:48.969264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:48.969264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:48.970056 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:48.972787 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 321/641] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:48.973555 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bd05c-f0b3-4eb5-be34-a17d8f63b170 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:49.000574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b49b94-ce4b-4b7a-931c-20222fe89b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:49.001254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b49b94-ce4b-4b7a-931c-20222fe89b1e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:49.001543 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b49b94-ce4b-4b7a-931c-20222fe89b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:49.001828 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b49b94-ce4b-4b7a-931c-20222fe89b1e None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:49.002473 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b49b94-ce4b-4b7a-931c-20222fe89b1e None admin] RBAC: Adding query filter params (name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:49.002777 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b49b94-ce4b-4b7a-931c-20222fe89b1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:49.004268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b49b94-ce4b-4b7a-931c-20222fe89b1e None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:49.012672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 321/642] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:14:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:49.013443 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b49b94-ce4b-4b7a-931c-20222fe89b1e None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:49.026413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:49.073458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bd05c-f0b3-4eb5-be34-a17d8f63b170 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:49.107580 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31396086-95cf-4d48-baf4-205bc3f1c91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:49.108198 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31396086-95cf-4d48-baf4-205bc3f1c91f None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:49.108480 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31396086-95cf-4d48-baf4-205bc3f1c91f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:49.108757 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31396086-95cf-4d48-baf4-205bc3f1c91f None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:49.109397 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31396086-95cf-4d48-baf4-205bc3f1c91f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:49.109701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31396086-95cf-4d48-baf4-205bc3f1c91f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:49.111106 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31396086-95cf-4d48-baf4-205bc3f1c91f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:49.116868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b49b94-ce4b-4b7a-931c-20222fe89b1e None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:49.122633 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 322/643] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:49.123442 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31396086-95cf-4d48-baf4-205bc3f1c91f None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:49.151230 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d3a9e-9bce-4120-bf9f-08dfd95f39c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:49.151864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d3a9e-9bce-4120-bf9f-08dfd95f39c8 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:49.152147 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d3a9e-9bce-4120-bf9f-08dfd95f39c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:49.152418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d3a9e-9bce-4120-bf9f-08dfd95f39c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:49.158984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d3a9e-9bce-4120-bf9f-08dfd95f39c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:49.160429 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d3a9e-9bce-4120-bf9f-08dfd95f39c8 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:49.173029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-cb6d3a9e-9bce-4120-bf9f-08dfd95f39c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:49.173029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:49.173029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:49.173029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:49.173029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:49.173029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:49.173029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:49.173029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:49.173029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:49.173029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:49.173029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:49.173029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:49.173029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:14:49.173029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:14:49.173029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:14:49.174073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:14:49.174073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:49.174073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:49.174073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:49.174073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:49.174073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:49.174073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:49.174073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:49.174073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:49.174073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:49.174073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:49.174073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:49.174073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:49.174073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:49.174073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:49.174073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:49.174073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:49.175365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:14:49.175365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:14:49.175365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:14:49.175365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:14:49.175365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:14:49.175365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:14:49.175365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:49.175365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:49.175365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 322/644] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:49.175365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d3a9e-9bce-4120-bf9f-08dfd95f39c8 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:49.199886 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:49.226644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31396086-95cf-4d48-baf4-205bc3f1c91f None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:49.240431 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:49.261704 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78df244-f391-4d15-b869-6cc4b0fc7b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:49.262312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78df244-f391-4d15-b869-6cc4b0fc7b4c None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:49.262596 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78df244-f391-4d15-b869-6cc4b0fc7b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:49.262901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78df244-f391-4d15-b869-6cc4b0fc7b4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:49.263625 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78df244-f391-4d15-b869-6cc4b0fc7b4c None admin] RBAC: Adding query filter params (role.id=0a83e40e2d64470a8489bf548e34dabd, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=709b9ae82dcd47408fc9a6d54a54428a) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:49.263938 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78df244-f391-4d15-b869-6cc4b0fc7b4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:49.265384 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78df244-f391-4d15-b869-6cc4b0fc7b4c None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:49.270965 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 323/645] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:14:49 2022] GET /identity/v3/role_assignments?user.id=709b9ae82dcd47408fc9a6d54a54428a&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=0a83e40e2d64470a8489bf548e34dabd => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:49.271725 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78df244-f391-4d15-b869-6cc4b0fc7b4c None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:49.272415 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d3a9e-9bce-4120-bf9f-08dfd95f39c8 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:49.322080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4399776-1203-425d-a7b7-56bc1f849d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:49.322714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4399776-1203-425d-a7b7-56bc1f849d4a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:49.323068 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4399776-1203-425d-a7b7-56bc1f849d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:49.323347 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4399776-1203-425d-a7b7-56bc1f849d4a None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:49.324073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4399776-1203-425d-a7b7-56bc1f849d4a None admin] RBAC: Adding query filter params (name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:49.324385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4399776-1203-425d-a7b7-56bc1f849d4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:49.325817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4399776-1203-425d-a7b7-56bc1f849d4a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:49.339663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 323/646] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:14:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:49.339663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4399776-1203-425d-a7b7-56bc1f849d4a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:49.391099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78df244-f391-4d15-b869-6cc4b0fc7b4c None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:49.430354 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:49.432281 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4aa07-0b4c-4601-936c-b81e3c7f52e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:49.432925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4aa07-0b4c-4601-936c-b81e3c7f52e9 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:49.433866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4aa07-0b4c-4601-936c-b81e3c7f52e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:49.433866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4aa07-0b4c-4601-936c-b81e3c7f52e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:49.444228 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4aa07-0b4c-4601-936c-b81e3c7f52e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:49.445835 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4aa07-0b4c-4601-936c-b81e3c7f52e9 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:49.449114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4399776-1203-425d-a7b7-56bc1f849d4a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:49.465276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-0ef4aa07-0b4c-4601-936c-b81e3c7f52e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:49.465276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:49.465276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:49.465276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:49.465276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:49.465276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:49.465276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:49.465276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:49.465276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:49.465276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:49.465276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:49.465276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:49.465276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:49.465276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:49.465276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:49.465276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:49.466412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:49.466412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:49.466412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:49.466412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:49.466412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:49.466412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:49.466412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:49.466412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:49.466412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:49.466412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:49.466412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:49.466412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:49.466412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:49.466412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:49.466412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:49.466412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:49.466412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:49.467483 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:49.467483 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:49.467483 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:49.467483 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 324/647] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:49.468219 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4aa07-0b4c-4601-936c-b81e3c7f52e9 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:49.494977 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4f8e8-7e75-4c4f-9634-320739bab95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:49.495856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4f8e8-7e75-4c4f-9634-320739bab95a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:49.496282 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4f8e8-7e75-4c4f-9634-320739bab95a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:49.496574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4f8e8-7e75-4c4f-9634-320739bab95a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:49.499986 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:49.505228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4f8e8-7e75-4c4f-9634-320739bab95a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:49.506668 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4f8e8-7e75-4c4f-9634-320739bab95a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:49.518454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-24d4f8e8-7e75-4c4f-9634-320739bab95a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 05 14:14:49.518454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:49.518454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:49.518454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:49.518454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:49.518454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:49.518454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:49.518454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:49.518454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:49.518454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:49.518454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:49.518454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:49.518454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:49.518454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:49.518454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:49.518454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:49.519483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:49.520452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:49.520452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:49.520452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:49.520452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:49.520452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:49.520452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:49.520452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:49.520452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:49.520452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 05 14:14:49.520452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:49.520452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 324/648] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:49.520452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4f8e8-7e75-4c4f-9634-320739bab95a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:49.577286 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4aa07-0b4c-4601-936c-b81e3c7f52e9 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:49.611195 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df2b5a-2826-4257-9575-52ba11b045d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:49.611842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df2b5a-2826-4257-9575-52ba11b045d4 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:49.612109 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df2b5a-2826-4257-9575-52ba11b045d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:49.612343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df2b5a-2826-4257-9575-52ba11b045d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:49.613030 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df2b5a-2826-4257-9575-52ba11b045d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:49.613337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df2b5a-2826-4257-9575-52ba11b045d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:49.614773 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df2b5a-2826-4257-9575-52ba11b045d4 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:49.627446 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 325/649] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:14:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:49.627967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4f8e8-7e75-4c4f-9634-320739bab95a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:49.628402 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df2b5a-2826-4257-9575-52ba11b045d4 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:49.666061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b663656-1e3d-437d-846e-1d255826f057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:49.666692 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b663656-1e3d-437d-846e-1d255826f057 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:49.667002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b663656-1e3d-437d-846e-1d255826f057 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:49.667274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b663656-1e3d-437d-846e-1d255826f057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:49.678314 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b663656-1e3d-437d-846e-1d255826f057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:49.679774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b663656-1e3d-437d-846e-1d255826f057 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:49.687297 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:49.695752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-7b663656-1e3d-437d-846e-1d255826f057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:49.695752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:49.695752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:49.695752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:49.695752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:49.695752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:49.695752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:49.695752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:49.695752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:49.695752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:49.695752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:49.695752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:49.695752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:49.695752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:49.695752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:49.695752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:49.707259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:49.707259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:49.707259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:49.707259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:49.707259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:49.707259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:49.707259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:49.707259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:49.707259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:49.707259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:49.707259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:49.707259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:49.707259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:49.707259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:49.707259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:49.707259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:49.707259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:49.708389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:49.708389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:49.708389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:49.708389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 325/650] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:49.708389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b663656-1e3d-437d-846e-1d255826f057 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:49.737240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:49.749558 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df2b5a-2826-4257-9575-52ba11b045d4 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:49.785997 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36670dc-d1d6-4cbe-ad66-ccdd7f264bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:49.786683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36670dc-d1d6-4cbe-ad66-ccdd7f264bda None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:49.787049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36670dc-d1d6-4cbe-ad66-ccdd7f264bda None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:49.787360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36670dc-d1d6-4cbe-ad66-ccdd7f264bda None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:49.788009 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36670dc-d1d6-4cbe-ad66-ccdd7f264bda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:49.788311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36670dc-d1d6-4cbe-ad66-ccdd7f264bda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:49.789688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36670dc-d1d6-4cbe-ad66-ccdd7f264bda None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:49.802098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 326/651] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:49.802892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36670dc-d1d6-4cbe-ad66-ccdd7f264bda None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:49.822747 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b663656-1e3d-437d-846e-1d255826f057 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:49.858461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e1165-82ed-479a-989c-fdad99997067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:49.859107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e1165-82ed-479a-989c-fdad99997067 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:49.859391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e1165-82ed-479a-989c-fdad99997067 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:49.859667 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e1165-82ed-479a-989c-fdad99997067 None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:49.860427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e1165-82ed-479a-989c-fdad99997067 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:49.860724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e1165-82ed-479a-989c-fdad99997067 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:49.862165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e1165-82ed-479a-989c-fdad99997067 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:49.884510 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 326/652] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Sep 5 14:14:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:49.884510 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e1165-82ed-479a-989c-fdad99997067 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:49.911166 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36670dc-d1d6-4cbe-ad66-ccdd7f264bda None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:49.943211 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:49.951489 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf83e5-6455-43ca-b4b8-559b5e7146b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:49.952175 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf83e5-6455-43ca-b4b8-559b5e7146b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:49.952487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf83e5-6455-43ca-b4b8-559b5e7146b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:49.952795 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf83e5-6455-43ca-b4b8-559b5e7146b2 None admin] PATH_INFO: `/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/0a85809d14cb449d848f22a3f39bf641/roles/0a83e40e2d64470a8489bf548e34dabd` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:49.968270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf83e5-6455-43ca-b4b8-559b5e7146b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9ca42deaed344eea726fe6e32356efc, user_id=0a85809d14cb449d848f22a3f39bf641, role_id=0a83e40e2d64470a8489bf548e34dabd)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:49.969863 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf83e5-6455-43ca-b4b8-559b5e7146b2 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:49.987477 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e1165-82ed-479a-989c-fdad99997067 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:50.021159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2869b47-c1ba-4332-90ba-244de04ad773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:50.021703 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2869b47-c1ba-4332-90ba-244de04ad773 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:50.021985 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2869b47-c1ba-4332-90ba-244de04ad773 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:50.022254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2869b47-c1ba-4332-90ba-244de04ad773 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:50.022956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2869b47-c1ba-4332-90ba-244de04ad773 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:50.023229 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:50.023334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2869b47-c1ba-4332-90ba-244de04ad773 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:50.025178 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2869b47-c1ba-4332-90ba-244de04ad773 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:50.037659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 327/653] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:14:49 2022] PUT /identity/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/0a85809d14cb449d848f22a3f39bf641/roles/0a83e40e2d64470a8489bf548e34dabd => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:14:50.037659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf83e5-6455-43ca-b4b8-559b5e7146b2 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:50.044945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 327/654] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:50.045942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2869b47-c1ba-4332-90ba-244de04ad773 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:50.175824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2869b47-c1ba-4332-90ba-244de04ad773 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:50.194601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:50.213768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf83e5-6455-43ca-b4b8-559b5e7146b2 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:50.223335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd350212-9283-4694-b88c-c4b502ba49a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:50.223502 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd350212-9283-4694-b88c-c4b502ba49a4 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:50.223781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd350212-9283-4694-b88c-c4b502ba49a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:50.224051 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd350212-9283-4694-b88c-c4b502ba49a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:50.259799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c5519-67c5-4fd9-9bfa-8f9e87ef95ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:50.260544 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c5519-67c5-4fd9-9bfa-8f9e87ef95ed None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:50.260829 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c5519-67c5-4fd9-9bfa-8f9e87ef95ed None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:50.261102 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c5519-67c5-4fd9-9bfa-8f9e87ef95ed None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:50.268958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c5519-67c5-4fd9-9bfa-8f9e87ef95ed None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:50.270531 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c5519-67c5-4fd9-9bfa-8f9e87ef95ed None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:50.283751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-cc5c5519-67c5-4fd9-9bfa-8f9e87ef95ed None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 14:14:50.283751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:50.283751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:50.283751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:50.283751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:50.283751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:50.283751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:50.283751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:50.283751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:50.283751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:50.283751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:50.283751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:50.283751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:50.283751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:50.283751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:50.283751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:50.284734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:50.285683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:50.285683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:50.285683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:50.285683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:50.285683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:50.285683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:50.285683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:50.285683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:50.285683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 14:14:50.285683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:50.285683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 328/655] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:50.286297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c5519-67c5-4fd9-9bfa-8f9e87ef95ed None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:50.310100 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd350212-9283-4694-b88c-c4b502ba49a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:50.311621 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd350212-9283-4694-b88c-c4b502ba49a4 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:50.371429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:50.422469 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 328/656] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:50.423524 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd350212-9283-4694-b88c-c4b502ba49a4 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:50.433439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c5519-67c5-4fd9-9bfa-8f9e87ef95ed None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:50.444152 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:50.480428 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f4424-467b-4295-80dc-cfd2fc30e062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:50.481827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f4424-467b-4295-80dc-cfd2fc30e062 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:50.482258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f4424-467b-4295-80dc-cfd2fc30e062 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:50.483016 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f4424-467b-4295-80dc-cfd2fc30e062 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:50.494342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f4424-467b-4295-80dc-cfd2fc30e062 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:50.496832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f4424-467b-4295-80dc-cfd2fc30e062 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:50.511154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-398f4424-467b-4295-80dc-cfd2fc30e062 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 14:14:50.511154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:50.511154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:50.511154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:50.511154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:50.511154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:50.511154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:50.511154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:50.511154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:50.511154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:50.511154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:50.511154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:50.511154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:50.511154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:50.511154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:50.511154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:50.512265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:50.513220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:50.513220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:50.513220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:50.513220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:50.513220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:50.513220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:50.513220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:50.513220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:50.513220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 14:14:50.513220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:50.517113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 329/657] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:50.517956 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f4424-467b-4295-80dc-cfd2fc30e062 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:50.544046 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd350212-9283-4694-b88c-c4b502ba49a4 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:50.584497 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9cc75-46c1-4140-8e06-e96f5f45ea6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:50.585047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9cc75-46c1-4140-8e06-e96f5f45ea6b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:50.585330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9cc75-46c1-4140-8e06-e96f5f45ea6b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:50.585602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9cc75-46c1-4140-8e06-e96f5f45ea6b None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:50.586337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9cc75-46c1-4140-8e06-e96f5f45ea6b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:50.586733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9cc75-46c1-4140-8e06-e96f5f45ea6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:50.588265 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9cc75-46c1-4140-8e06-e96f5f45ea6b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:50.611230 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 329/658] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Sep 5 14:14:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:50.611916 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9cc75-46c1-4140-8e06-e96f5f45ea6b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:50.638304 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f4424-467b-4295-80dc-cfd2fc30e062 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:50.656457 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:50.674180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c5aad-3922-49df-bb3f-685c539985cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:50.674865 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c5aad-3922-49df-bb3f-685c539985cf None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:50.675193 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c5aad-3922-49df-bb3f-685c539985cf None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:50.675509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c5aad-3922-49df-bb3f-685c539985cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:50.686291 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c5aad-3922-49df-bb3f-685c539985cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:50.687795 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c5aad-3922-49df-bb3f-685c539985cf None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:50.704333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-2b8c5aad-3922-49df-bb3f-685c539985cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:50.704333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:50.704333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:50.704333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:50.704333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:50.704333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:50.704333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:50.704333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:50.704333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:50.704333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:50.704333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:50.704333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:50.704333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:50.704333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:50.704333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:50.704333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:50.705492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:50.705492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:50.705492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:50.705492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:50.705492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:50.705492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:50.705492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:50.705492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:50.705492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:50.705492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:50.705492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:50.705492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:50.705492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:50.705492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:50.705492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:50.705492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:50.705492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:50.706533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:50.706533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:50.706533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:50.706533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 330/659] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:50.706533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c5aad-3922-49df-bb3f-685c539985cf None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:50.716862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:50.743755 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9cc75-46c1-4140-8e06-e96f5f45ea6b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:50.780806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d366d9-e7ec-4942-af61-774871cd53bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:50.781613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d366d9-e7ec-4942-af61-774871cd53bb None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:50.781899 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d366d9-e7ec-4942-af61-774871cd53bb None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:50.782178 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d366d9-e7ec-4942-af61-774871cd53bb None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:50.782943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d366d9-e7ec-4942-af61-774871cd53bb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:50.783255 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d366d9-e7ec-4942-af61-774871cd53bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:50.784808 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d366d9-e7ec-4942-af61-774871cd53bb None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:50.809477 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 330/660] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Sep 5 14:14:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:50.810376 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d366d9-e7ec-4942-af61-774871cd53bb None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:50.836628 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c5aad-3922-49df-bb3f-685c539985cf None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:50.881896 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe398028-b23d-41b8-b3d6-8b544b62ff72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:50.883063 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe398028-b23d-41b8-b3d6-8b544b62ff72 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:50.883567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe398028-b23d-41b8-b3d6-8b544b62ff72 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:50.884654 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe398028-b23d-41b8-b3d6-8b544b62ff72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:50.921961 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d366d9-e7ec-4942-af61-774871cd53bb None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:50.937117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:50.964110 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:50.968190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe398028-b23d-41b8-b3d6-8b544b62ff72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:50.969668 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe398028-b23d-41b8-b3d6-8b544b62ff72 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:50.970925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c2f2d-8ef0-4bbb-8792-a209ee85e04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:50.971979 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c2f2d-8ef0-4bbb-8792-a209ee85e04e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:50.973245 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c2f2d-8ef0-4bbb-8792-a209ee85e04e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:50.973731 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c2f2d-8ef0-4bbb-8792-a209ee85e04e None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:50.974573 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c2f2d-8ef0-4bbb-8792-a209ee85e04e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:50.975239 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c2f2d-8ef0-4bbb-8792-a209ee85e04e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:50.976798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c2f2d-8ef0-4bbb-8792-a209ee85e04e None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:50.994519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 331/661] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:50.995890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c2f2d-8ef0-4bbb-8792-a209ee85e04e None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:51.082671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 331/662] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:51.082671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe398028-b23d-41b8-b3d6-8b544b62ff72 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:51.136683 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c2f2d-8ef0-4bbb-8792-a209ee85e04e None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:51.162763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:51.172235 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed781dd9-e5f2-44aa-9710-02aeff88a61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:51.173026 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed781dd9-e5f2-44aa-9710-02aeff88a61d None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:51.178947 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed781dd9-e5f2-44aa-9710-02aeff88a61d None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:51.178947 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed781dd9-e5f2-44aa-9710-02aeff88a61d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:51.199355 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe398028-b23d-41b8-b3d6-8b544b62ff72 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:51.236890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:51.240425 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d5d5c-3ebc-496a-89c6-5edc603960cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:51.241232 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d5d5c-3ebc-496a-89c6-5edc603960cc None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:51.241701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d5d5c-3ebc-496a-89c6-5edc603960cc None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:51.242144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d5d5c-3ebc-496a-89c6-5edc603960cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:51.248106 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed781dd9-e5f2-44aa-9710-02aeff88a61d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:51.249649 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed781dd9-e5f2-44aa-9710-02aeff88a61d None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:51.250369 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d5d5c-3ebc-496a-89c6-5edc603960cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:51.251996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d5d5c-3ebc-496a-89c6-5edc603960cc None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:51.264846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-0d0d5d5c-3ebc-496a-89c6-5edc603960cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:51.264846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:51.264846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:51.264846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:51.264846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:51.264846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:51.264846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:51.264846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:51.264846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:51.264846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:51.264846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:51.264846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:51.264846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:14:51.264846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:14:51.264846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:14:51.265892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:14:51.265892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:51.265892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:51.265892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:51.265892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:51.265892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:51.265892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:51.265892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:51.265892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:51.265892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:51.265892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:51.265892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:51.265892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:51.265892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:51.265892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:51.265892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:51.265892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:51.267080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:14:51.267080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:14:51.267080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:14:51.267080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:14:51.267080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:14:51.267080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:14:51.267080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:51.267080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:51.268900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 332/663] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:51.269811 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d5d5c-3ebc-496a-89c6-5edc603960cc None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:51.359029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 332/664] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:51.360279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09018271-0839-4300-8fa4-83787580b5a1 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:51.360566 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09018271-0839-4300-8fa4-83787580b5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:51.360836 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09018271-0839-4300-8fa4-83787580b5a1 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:51.366413 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 333/665] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:51.367186 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09018271-0839-4300-8fa4-83787580b5a1 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:51.382965 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d5d5c-3ebc-496a-89c6-5edc603960cc None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:51.410154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:51.416505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d2e7c-5d7d-41ae-978c-ff1a7e66afd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:51.423406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d2e7c-5d7d-41ae-978c-ff1a7e66afd1 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:51.423406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d2e7c-5d7d-41ae-978c-ff1a7e66afd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:51.423406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d2e7c-5d7d-41ae-978c-ff1a7e66afd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:51.427921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d2e7c-5d7d-41ae-978c-ff1a7e66afd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:51.429474 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d2e7c-5d7d-41ae-978c-ff1a7e66afd1 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:51.444247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-690d2e7c-5d7d-41ae-978c-ff1a7e66afd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:51.444247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:51.444247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:51.444247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:51.444247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:51.444247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:51.444247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:51.444247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:51.444247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:51.444247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:51.444247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:51.444247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:51.444247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:51.444247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:51.444247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:51.444247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:51.445413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:51.445413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:51.445413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:51.445413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:51.445413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:51.445413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:51.445413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:51.445413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:51.445413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:51.445413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:51.445413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:51.445413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:51.445413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:51.445413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:51.445413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:51.445413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:51.445413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:51.446542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:51.446542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:51.446542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:51.446542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 333/666] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:51.446542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d2e7c-5d7d-41ae-978c-ff1a7e66afd1 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:51.468796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:51.479361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09018271-0839-4300-8fa4-83787580b5a1 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:51.516702 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e1961-137d-4438-90b1-90bf0c9a85c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:51.517338 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e1961-137d-4438-90b1-90bf0c9a85c3 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:51.517619 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e1961-137d-4438-90b1-90bf0c9a85c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:51.517908 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e1961-137d-4438-90b1-90bf0c9a85c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:51.518577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e1961-137d-4438-90b1-90bf0c9a85c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:51.519252 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e1961-137d-4438-90b1-90bf0c9a85c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:51.520354 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e1961-137d-4438-90b1-90bf0c9a85c3 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:51.533532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 334/667] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:14:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:51.534557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb700e-e822-450e-a4c8-726771ab6b23 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:51.534879 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb700e-e822-450e-a4c8-726771ab6b23 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:51.535263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb700e-e822-450e-a4c8-726771ab6b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:51.553242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-9abb700e-e822-450e-a4c8-726771ab6b23 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:51.554430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-9abb700e-e822-450e-a4c8-726771ab6b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:51.556764 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d2e7c-5d7d-41ae-978c-ff1a7e66afd1 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:51.593913 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd2aa7-e446-46fa-9445-179e6467822a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:51.594555 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd2aa7-e446-46fa-9445-179e6467822a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:51.594858 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd2aa7-e446-46fa-9445-179e6467822a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:51.595155 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd2aa7-e446-46fa-9445-179e6467822a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:51.606997 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd2aa7-e446-46fa-9445-179e6467822a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:51.608582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd2aa7-e446-46fa-9445-179e6467822a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:51.625971 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 335/668] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:51.625971 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb700e-e822-450e-a4c8-726771ab6b23 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:51.626187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-6bbd2aa7-e446-46fa-9445-179e6467822a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:51.626187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:51.626187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:51.626187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:51.626187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:51.626187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:51.626187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:51.626187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:51.626187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:51.626187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:51.626187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:51.626187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:51.626187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:51.626187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:51.626187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:51.626187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:51.629831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:51.629831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:51.629831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:51.629831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:51.629831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:51.629831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:51.629831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:51.629831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:51.629831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:51.629831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:51.629831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:51.629831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:51.629831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:51.629831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:51.629831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:51.629831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:51.629831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:51.630974 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:51.630974 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:51.630974 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:51.630974 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 334/669] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:51.630974 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd2aa7-e446-46fa-9445-179e6467822a None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:51.672144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:51.727179 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:51.749313 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd2aa7-e446-46fa-9445-179e6467822a None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:51.754723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb700e-e822-450e-a4c8-726771ab6b23 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:51.789150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9a99a-4e2a-4e70-b958-bccb2e6d1547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:51.789762 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9a99a-4e2a-4e70-b958-bccb2e6d1547 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:51.790203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9a99a-4e2a-4e70-b958-bccb2e6d1547 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:51.790374 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9a99a-4e2a-4e70-b958-bccb2e6d1547 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:51.791129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9a99a-4e2a-4e70-b958-bccb2e6d1547 None admin] RBAC: Adding query filter params (role.id=0a83e40e2d64470a8489bf548e34dabd, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=735986478c354ed196c601f12f3ac377) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:51.791438 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9a99a-4e2a-4e70-b958-bccb2e6d1547 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:51.793013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9a99a-4e2a-4e70-b958-bccb2e6d1547 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:51.798217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41757ab1-6cdf-4104-badf-89b9fe260061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:51.799055 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41757ab1-6cdf-4104-badf-89b9fe260061 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:51.806403 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 335/670] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:14:51 2022] GET /identity/v3/role_assignments?user.id=735986478c354ed196c601f12f3ac377&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=0a83e40e2d64470a8489bf548e34dabd => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:51.806403 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9a99a-4e2a-4e70-b958-bccb2e6d1547 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:51.807243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41757ab1-6cdf-4104-badf-89b9fe260061 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:51.807747 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41757ab1-6cdf-4104-badf-89b9fe260061 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:51.808616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41757ab1-6cdf-4104-badf-89b9fe260061 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:51.809059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41757ab1-6cdf-4104-badf-89b9fe260061 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:51.810800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41757ab1-6cdf-4104-badf-89b9fe260061 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:51.832925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 336/671] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:51.839583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41757ab1-6cdf-4104-badf-89b9fe260061 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:51.911470 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:51.917344 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9a99a-4e2a-4e70-b958-bccb2e6d1547 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:51.970246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524165e-e7c0-4336-a29b-88c9001c0a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:51.971350 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524165e-e7c0-4336-a29b-88c9001c0a47 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:51.971643 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524165e-e7c0-4336-a29b-88c9001c0a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:51.972077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524165e-e7c0-4336-a29b-88c9001c0a47 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:51.972858 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524165e-e7c0-4336-a29b-88c9001c0a47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:51.973169 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524165e-e7c0-4336-a29b-88c9001c0a47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:51.975153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524165e-e7c0-4336-a29b-88c9001c0a47 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:51.990253 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41757ab1-6cdf-4104-badf-89b9fe260061 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:51.990621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 336/672] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:51.991619 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524165e-e7c0-4336-a29b-88c9001c0a47 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:52.029549 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b887ff5-1df7-4f3c-8a38-83305e4bdec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:52.033752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b887ff5-1df7-4f3c-8a38-83305e4bdec8 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:52.033752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b887ff5-1df7-4f3c-8a38-83305e4bdec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:52.033752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b887ff5-1df7-4f3c-8a38-83305e4bdec8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:52.043716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b887ff5-1df7-4f3c-8a38-83305e4bdec8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:52.045370 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b887ff5-1df7-4f3c-8a38-83305e4bdec8 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:52.059229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-7b887ff5-1df7-4f3c-8a38-83305e4bdec8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:52.059229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:52.059229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:52.059229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:52.059229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:52.059229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:52.059229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:52.059229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:52.059229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:52.059229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:52.059229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:52.059229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:52.059229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:14:52.059229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:14:52.059229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:14:52.060316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:14:52.060316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:52.060316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:52.060316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:52.060316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:52.060316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:52.060316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:52.060316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:52.060316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:52.060316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:52.060316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:52.060316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:52.060316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:52.060316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:52.060316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:52.060316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:52.060316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:52.061285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:14:52.061285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:14:52.061285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:14:52.061285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:14:52.061285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:14:52.061285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:14:52.061285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:52.061285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:52.063616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 337/673] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:14:52.064629 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b887ff5-1df7-4f3c-8a38-83305e4bdec8 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:52.067729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:52.109843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524165e-e7c0-4336-a29b-88c9001c0a47 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:52.151716 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd380e6b-1ec5-45b0-97e7-d7f613fe1e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:52.152356 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd380e6b-1ec5-45b0-97e7-d7f613fe1e7a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:52.152648 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd380e6b-1ec5-45b0-97e7-d7f613fe1e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:52.152923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd380e6b-1ec5-45b0-97e7-d7f613fe1e7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:52.164505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd380e6b-1ec5-45b0-97e7-d7f613fe1e7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:52.170120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-dd380e6b-1ec5-45b0-97e7-d7f613fe1e7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:14:52.170120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:52.170120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:52.170120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:52.170120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:52.170120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:52.170120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:52.170120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:52.170120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:52.170120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:52.170120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:52.170120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:52.170120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:52.170120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:52.170120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:52.170120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:52.171147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:52.171147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:52.171147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:52.171147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:52.171147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:52.171147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:52.171147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:52.171147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:52.171147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:52.171147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:52.171147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:52.171147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:52.171147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:52.171147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:52.171147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:52.171147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:52.172011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:52.172011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:52.172011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:52.172011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:52.172011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:52.172011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:52.172011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:52.172011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:52.172011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:52.172011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:52.172011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:14:52.172011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:52.174995 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 337/674] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:14:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:52.176325 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd380e6b-1ec5-45b0-97e7-d7f613fe1e7a None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:52.185400 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b887ff5-1df7-4f3c-8a38-83305e4bdec8 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:52.197347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:52.222121 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66a9c4-fa7c-4eb1-a8db-f95ad175c30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:52.222937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66a9c4-fa7c-4eb1-a8db-f95ad175c30a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:52.223219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66a9c4-fa7c-4eb1-a8db-f95ad175c30a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:52.223499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66a9c4-fa7c-4eb1-a8db-f95ad175c30a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:52.231044 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66a9c4-fa7c-4eb1-a8db-f95ad175c30a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:52.232494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66a9c4-fa7c-4eb1-a8db-f95ad175c30a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:52.243909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-ac66a9c4-fa7c-4eb1-a8db-f95ad175c30a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:52.243909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:52.243909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:52.243909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:52.243909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:52.243909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:52.243909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:52.243909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:52.243909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:52.243909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:52.243909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:52.243909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:52.243909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:14:52.243909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:14:52.243909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:14:52.245080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:14:52.245080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:52.245080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:52.245080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:52.245080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:52.245080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:52.245080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:52.245080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:52.245080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:52.245080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:52.245080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:52.245080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:52.245080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:52.245080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:52.245080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:52.245080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:52.245080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:52.246492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:14:52.246492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:14:52.246492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:14:52.246492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:14:52.246492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:14:52.246492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:14:52.246492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:52.246492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:52.249845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 338/675] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:52.251101 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225c630-2c9f-4d64-b23f-b4584832a474 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:52.251394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225c630-2c9f-4d64-b23f-b4584832a474 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:52.251676 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225c630-2c9f-4d64-b23f-b4584832a474 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:52.253087 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 339/676] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:52.253616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225c630-2c9f-4d64-b23f-b4584832a474 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:52.287785 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd380e6b-1ec5-45b0-97e7-d7f613fe1e7a None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:52.325246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:52.326810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637119a-c892-4c66-99b0-dbd6efd3c720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:52.327444 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637119a-c892-4c66-99b0-dbd6efd3c720 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:52.327726 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637119a-c892-4c66-99b0-dbd6efd3c720 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:52.328012 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637119a-c892-4c66-99b0-dbd6efd3c720 None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:52.328685 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637119a-c892-4c66-99b0-dbd6efd3c720 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:52.328989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637119a-c892-4c66-99b0-dbd6efd3c720 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:52.330928 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637119a-c892-4c66-99b0-dbd6efd3c720 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:52.346861 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 338/677] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:14:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:52.346861 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637119a-c892-4c66-99b0-dbd6efd3c720 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:52.373991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225c630-2c9f-4d64-b23f-b4584832a474 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:52.410609 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf97dce-485d-4513-a6cf-d63a999dbafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:52.411245 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf97dce-485d-4513-a6cf-d63a999dbafa None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:52.411524 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf97dce-485d-4513-a6cf-d63a999dbafa None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:52.411799 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf97dce-485d-4513-a6cf-d63a999dbafa None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:52.412454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf97dce-485d-4513-a6cf-d63a999dbafa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:52.412755 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf97dce-485d-4513-a6cf-d63a999dbafa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:52.414119 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf97dce-485d-4513-a6cf-d63a999dbafa None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:52.427456 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 340/678] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:52.427456 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0873c22-9c72-42e4-93f0-4e25b9585e07 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:52.427456 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0873c22-9c72-42e4-93f0-4e25b9585e07 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:52.427456 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0873c22-9c72-42e4-93f0-4e25b9585e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:52.430344 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:52.444631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-b0873c22-9c72-42e4-93f0-4e25b9585e07 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:52.445940 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b0873c22-9c72-42e4-93f0-4e25b9585e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:52.451743 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637119a-c892-4c66-99b0-dbd6efd3c720 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:52.485871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757e60a-2e43-4f79-aca0-9c778818826e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:52.486479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757e60a-2e43-4f79-aca0-9c778818826e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:52.486761 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757e60a-2e43-4f79-aca0-9c778818826e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:52.487065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757e60a-2e43-4f79-aca0-9c778818826e None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:52.487755 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757e60a-2e43-4f79-aca0-9c778818826e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:52.488058 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757e60a-2e43-4f79-aca0-9c778818826e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:52.489503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757e60a-2e43-4f79-aca0-9c778818826e None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:52.506729 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 339/679] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:14:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:52.506729 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757e60a-2e43-4f79-aca0-9c778818826e None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:52.522697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 341/680] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:52.523626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0873c22-9c72-42e4-93f0-4e25b9585e07 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:52.583536 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:52.619837 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757e60a-2e43-4f79-aca0-9c778818826e None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:52.631263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0873c22-9c72-42e4-93f0-4e25b9585e07 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:52.657928 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91af61-836a-4b5d-ad74-ae50c23d56fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:52.658771 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91af61-836a-4b5d-ad74-ae50c23d56fc None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:52.659306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91af61-836a-4b5d-ad74-ae50c23d56fc None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:52.659757 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91af61-836a-4b5d-ad74-ae50c23d56fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:52.660827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b91af61-836a-4b5d-ad74-ae50c23d56fc None admin] RBAC: Adding query filter params (role.id=0a83e40e2d64470a8489bf548e34dabd, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=5c73ca2785334fc496b5227d3be655c7) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:52.661285 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b91af61-836a-4b5d-ad74-ae50c23d56fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:52.663108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b91af61-836a-4b5d-ad74-ae50c23d56fc None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:52.668153 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb97f28-1c0d-4efc-af6d-53b8e9e802fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:52.668902 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb97f28-1c0d-4efc-af6d-53b8e9e802fb None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:52.669245 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb97f28-1c0d-4efc-af6d-53b8e9e802fb None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:52.669584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb97f28-1c0d-4efc-af6d-53b8e9e802fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:52.675178 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 340/681] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:14:52 2022] GET /identity/v3/role_assignments?user.id=5c73ca2785334fc496b5227d3be655c7&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=0a83e40e2d64470a8489bf548e34dabd => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:52.676626 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91af61-836a-4b5d-ad74-ae50c23d56fc None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:52.682887 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:52.683037 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb97f28-1c0d-4efc-af6d-53b8e9e802fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:52.684908 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb97f28-1c0d-4efc-af6d-53b8e9e802fb None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:52.701292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-1cb97f28-1c0d-4efc-af6d-53b8e9e802fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:52.701292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:52.701292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:52.701292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:52.701292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:52.701292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:52.701292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:52.701292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:52.701292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:52.701292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:52.701292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:52.701292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:52.701292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:52.701292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:52.701292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:52.701292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:52.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:52.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:52.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:52.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:52.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:52.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:52.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:52.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:52.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:52.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:52.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:52.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:52.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:52.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:52.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:52.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:52.702356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:52.703319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:52.703319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:52.703319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:52.709671 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 342/682] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:52.709671 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb97f28-1c0d-4efc-af6d-53b8e9e802fb None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:52.826976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb97f28-1c0d-4efc-af6d-53b8e9e802fb None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:52.835019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:52.840781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91af61-836a-4b5d-ad74-ae50c23d56fc None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:52.868396 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fb2f4-cf2c-4567-ba91-3d086db6c946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:52.869026 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fb2f4-cf2c-4567-ba91-3d086db6c946 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:52.869307 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fb2f4-cf2c-4567-ba91-3d086db6c946 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:52.869582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fb2f4-cf2c-4567-ba91-3d086db6c946 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:52.870233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4fb2f4-cf2c-4567-ba91-3d086db6c946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:52.870815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4fb2f4-cf2c-4567-ba91-3d086db6c946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:52.872188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4fb2f4-cf2c-4567-ba91-3d086db6c946 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:52.890714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 343/683] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:52.892003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fb2f4-cf2c-4567-ba91-3d086db6c946 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:52.892440 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c18ac8-0c23-4bcd-96b6-9edf7fd5771e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:52.893475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c18ac8-0c23-4bcd-96b6-9edf7fd5771e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:52.893475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c18ac8-0c23-4bcd-96b6-9edf7fd5771e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:52.893641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c18ac8-0c23-4bcd-96b6-9edf7fd5771e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:52.894319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c18ac8-0c23-4bcd-96b6-9edf7fd5771e None admin] RBAC: Adding query filter params (role.id=0a83e40e2d64470a8489bf548e34dabd, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=e49ac59302cb42f7942954d3859d3675) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:52.894844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c18ac8-0c23-4bcd-96b6-9edf7fd5771e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:52.897067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c18ac8-0c23-4bcd-96b6-9edf7fd5771e None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:52.902933 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 341/684] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:14:52 2022] GET /identity/v3/role_assignments?user.id=e49ac59302cb42f7942954d3859d3675&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=0a83e40e2d64470a8489bf548e34dabd => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:52.903798 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c18ac8-0c23-4bcd-96b6-9edf7fd5771e None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:53.014023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:53.038584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fb2f4-cf2c-4567-ba91-3d086db6c946 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:53.050075 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c18ac8-0c23-4bcd-96b6-9edf7fd5771e None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:53.090313 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1ff46-d2a4-492d-8880-30ca9b12e3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:53.090990 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1ff46-d2a4-492d-8880-30ca9b12e3e5 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:53.091284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1ff46-d2a4-492d-8880-30ca9b12e3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:53.094319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1ff46-d2a4-492d-8880-30ca9b12e3e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:53.101352 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2175c40-bfab-4a88-aa5c-8d52448014b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:53.101998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2175c40-bfab-4a88-aa5c-8d52448014b5 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:53.102287 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2175c40-bfab-4a88-aa5c-8d52448014b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:53.103275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2175c40-bfab-4a88-aa5c-8d52448014b5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:53.104646 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1ff46-d2a4-492d-8880-30ca9b12e3e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:53.109608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-c6a1ff46-d2a4-492d-8880-30ca9b12e3e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:53.109608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:53.109608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:53.109608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:53.109608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:53.109608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:53.109608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:53.109608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:53.109608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:53.109608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:53.109608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:53.109608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:53.109608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:53.109608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:53.109608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:53.109608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:53.110544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:53.110544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:53.110544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:53.110544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:53.110544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:53.110544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:53.110544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:53.110544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:53.110544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:53.110544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:53.110544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:53.110544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:53.110544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:53.110544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:53.110544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:53.110544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:53.111581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:53.111581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:53.111581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:53.111581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:53.111581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:53.111581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:53.111581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:53.111581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:53.111581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:53.111581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:53.111581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:53.111581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:53.112799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2175c40-bfab-4a88-aa5c-8d52448014b5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:53.112895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 344/685] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:53.112985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2175c40-bfab-4a88-aa5c-8d52448014b5 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:53.123604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1ff46-d2a4-492d-8880-30ca9b12e3e5 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:53.124939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-f2175c40-bfab-4a88-aa5c-8d52448014b5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:14:53.124939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:53.124939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:53.124939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:53.124939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:53.124939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:53.124939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:53.124939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:53.124939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:53.124939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:53.124939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:53.124939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:53.124939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:53.124939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:53.124939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:53.124939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:53.125876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:53.127807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:53.127807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:53.127807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:53.127807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:53.127807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:53.127807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:53.127807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:53.127807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:53.127807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:14:53.127807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:53.127807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 342/686] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 5 14:14:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:53.134069 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2175c40-bfab-4a88-aa5c-8d52448014b5 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:53.165546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:53.260260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2175c40-bfab-4a88-aa5c-8d52448014b5 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:53.264616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1ff46-d2a4-492d-8880-30ca9b12e3e5 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:53.287656 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:53.297120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9627681-3a0a-462d-aa6a-90ef8d25606e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:53.305862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9627681-3a0a-462d-aa6a-90ef8d25606e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:53.305862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9627681-3a0a-462d-aa6a-90ef8d25606e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:53.305862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9627681-3a0a-462d-aa6a-90ef8d25606e None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:53.305862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9627681-3a0a-462d-aa6a-90ef8d25606e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:53.305862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9627681-3a0a-462d-aa6a-90ef8d25606e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:53.305862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9627681-3a0a-462d-aa6a-90ef8d25606e None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:53.318527 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e78316-333a-4333-8edd-a8e535a2295e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:53.319230 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e78316-333a-4333-8edd-a8e535a2295e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:53.319507 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e78316-333a-4333-8edd-a8e535a2295e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:53.319776 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e78316-333a-4333-8edd-a8e535a2295e None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:53.320448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e78316-333a-4333-8edd-a8e535a2295e None admin] RBAC: Adding query filter params (name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:53.320747 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e78316-333a-4333-8edd-a8e535a2295e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:53.322118 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e78316-333a-4333-8edd-a8e535a2295e None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:53.329440 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 343/687] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 5 14:14:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:53.335152 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9627681-3a0a-462d-aa6a-90ef8d25606e None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:53.341423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 345/688] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:14:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:53.348942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e78316-333a-4333-8edd-a8e535a2295e None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:53.499178 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:53.519501 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e78316-333a-4333-8edd-a8e535a2295e None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:53.531785 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9627681-3a0a-462d-aa6a-90ef8d25606e None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:53.572254 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d29d26-0208-4555-a322-450a931ed475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:53.572875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d29d26-0208-4555-a322-450a931ed475 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:53.573160 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d29d26-0208-4555-a322-450a931ed475 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:53.573437 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d29d26-0208-4555-a322-450a931ed475 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:53.591297 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a76a13-0f3c-4600-a951-5e8cf370e6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:53.591946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a76a13-0f3c-4600-a951-5e8cf370e6ef None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:53.592228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a76a13-0f3c-4600-a951-5e8cf370e6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:53.592502 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a76a13-0f3c-4600-a951-5e8cf370e6ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:53.612865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a76a13-0f3c-4600-a951-5e8cf370e6ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:53.614336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a76a13-0f3c-4600-a951-5e8cf370e6ef None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:53.642717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-c6a76a13-0f3c-4600-a951-5e8cf370e6ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:53.642717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:53.642717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:53.642717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:53.642717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:53.642717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:53.642717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:53.642717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:53.642717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:53.642717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:53.642717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:53.642717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:53.642717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:53.642717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:53.642717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:53.642717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:53.643759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:53.643759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:53.643759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:53.643759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:53.643759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:53.643759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:53.643759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:53.643759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:53.643759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:53.643759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:53.643759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:53.643759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:53.643759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:53.643759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:53.643759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:53.643759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:53.643759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:53.645243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:53.645243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:53.645243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:53.645243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 346/689] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:53.650316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a76a13-0f3c-4600-a951-5e8cf370e6ef None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:53.675885 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:53.710114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d29d26-0208-4555-a322-450a931ed475 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:53.711639 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d29d26-0208-4555-a322-450a931ed475 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:53.842558 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a76a13-0f3c-4600-a951-5e8cf370e6ef None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:53.882283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6daef6-80e2-461d-8203-32b9c647697b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:53.882955 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6daef6-80e2-461d-8203-32b9c647697b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:53.883459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6daef6-80e2-461d-8203-32b9c647697b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:53.883767 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6daef6-80e2-461d-8203-32b9c647697b None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:53.884659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6daef6-80e2-461d-8203-32b9c647697b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:53.885062 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6daef6-80e2-461d-8203-32b9c647697b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:53.886612 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6daef6-80e2-461d-8203-32b9c647697b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:53.897210 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:53.902803 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 347/690] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:53.907873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6daef6-80e2-461d-8203-32b9c647697b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:53.909643 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 344/691] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 14:14:53.915343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d29d26-0208-4555-a322-450a931ed475 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:54.034031 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:54.051475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d29d26-0208-4555-a322-450a931ed475 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:54.089616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013e6fe-cffa-4db6-ae2c-7398064384e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:54.090664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013e6fe-cffa-4db6-ae2c-7398064384e7 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:54.091172 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013e6fe-cffa-4db6-ae2c-7398064384e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:54.091735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013e6fe-cffa-4db6-ae2c-7398064384e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:54.104338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013e6fe-cffa-4db6-ae2c-7398064384e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:54.106094 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013e6fe-cffa-4db6-ae2c-7398064384e7 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:54.112856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6daef6-80e2-461d-8203-32b9c647697b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:54.123007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-5013e6fe-cffa-4db6-ae2c-7398064384e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:54.123007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:54.123007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:54.123007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:54.123007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:54.123007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:54.123007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:54.123007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:54.123007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:54.123007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:54.123007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:54.123007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:54.123007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:54.123007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:54.123007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:54.123007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:54.124065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:54.124065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:54.124065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:54.124065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:54.124065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:54.124065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:54.124065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:54.124065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:54.124065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:54.124065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:54.124065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:54.124065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:54.124065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:54.124065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:54.124065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:54.124065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:54.124065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:54.124984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:54.124984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:54.124984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:54.127360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 345/692] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:14:54.128873 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f55990-a5a1-4dcb-978d-4d53a402d2fd None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:54.129302 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f55990-a5a1-4dcb-978d-4d53a402d2fd None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:54.129740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f55990-a5a1-4dcb-978d-4d53a402d2fd None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:54.131646 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 346/693] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:54.132511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f55990-a5a1-4dcb-978d-4d53a402d2fd None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:54.159260 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527de43b-a098-498f-874d-a9a56535ef45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:54.159958 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527de43b-a098-498f-874d-a9a56535ef45 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:54.160216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527de43b-a098-498f-874d-a9a56535ef45 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:54.160497 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527de43b-a098-498f-874d-a9a56535ef45 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:54.170162 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527de43b-a098-498f-874d-a9a56535ef45 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:54.171626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527de43b-a098-498f-874d-a9a56535ef45 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:54.184258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-527de43b-a098-498f-874d-a9a56535ef45 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 14:14:54.184258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:54.184258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:54.184258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:54.184258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:54.184258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:54.184258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:54.184258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:54.184258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:54.184258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:54.184258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:54.184258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:54.184258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:54.184258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:54.184258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:54.184258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:54.185451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:54.186525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:54.186525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:54.186525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:54.186525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:54.186525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:54.186525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:54.186525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:54.186525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:54.186525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 14:14:54.186525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:54.186525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 348/694] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:14:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:14:54.191211 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bad65b-7b3a-40bb-9aec-304b224ba604 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:54.191552 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bad65b-7b3a-40bb-9aec-304b224ba604 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:54.191798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bad65b-7b3a-40bb-9aec-304b224ba604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:54.211042 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-48bad65b-7b3a-40bb-9aec-304b224ba604 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:54.212848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-48bad65b-7b3a-40bb-9aec-304b224ba604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:54.249241 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f55990-a5a1-4dcb-978d-4d53a402d2fd None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:54.272021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:54.283526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 349/695] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:54.289827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bad65b-7b3a-40bb-9aec-304b224ba604 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:54.290670 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb2d3e-f10a-473c-acb2-d8995ae14a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:54.290670 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb2d3e-f10a-473c-acb2-d8995ae14a62 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:54.290670 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb2d3e-f10a-473c-acb2-d8995ae14a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:54.290670 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb2d3e-f10a-473c-acb2-d8995ae14a62 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:54.291011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb2d3e-f10a-473c-acb2-d8995ae14a62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:54.291259 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb2d3e-f10a-473c-acb2-d8995ae14a62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:54.292882 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb2d3e-f10a-473c-acb2-d8995ae14a62 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:54.295777 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:54.307620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 347/696] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:54.316008 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb2d3e-f10a-473c-acb2-d8995ae14a62 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:54.400581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bad65b-7b3a-40bb-9aec-304b224ba604 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:54.440952 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81143488-c709-4129-a81f-2bb5e12da90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:54.441671 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81143488-c709-4129-a81f-2bb5e12da90f None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:54.442049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81143488-c709-4129-a81f-2bb5e12da90f None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:54.442498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81143488-c709-4129-a81f-2bb5e12da90f None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:54.446831 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81143488-c709-4129-a81f-2bb5e12da90f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:54.446831 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81143488-c709-4129-a81f-2bb5e12da90f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:54.446831 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81143488-c709-4129-a81f-2bb5e12da90f None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:54.465580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 350/697] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Sep 5 14:14:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:54.471374 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81143488-c709-4129-a81f-2bb5e12da90f None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:54.528131 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:54.542250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb2d3e-f10a-473c-acb2-d8995ae14a62 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:54.587090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81143488-c709-4129-a81f-2bb5e12da90f None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:54.626371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ffccf-d19f-4498-b7e4-77c54acdcc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:54.627403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ffccf-d19f-4498-b7e4-77c54acdcc3a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:54.627871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ffccf-d19f-4498-b7e4-77c54acdcc3a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:54.628287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ffccf-d19f-4498-b7e4-77c54acdcc3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:54.643582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a0ce8-7e12-4b04-8eea-1706ae925403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:54.644631 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a0ce8-7e12-4b04-8eea-1706ae925403 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:54.644769 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a0ce8-7e12-4b04-8eea-1706ae925403 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:54.645144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a0ce8-7e12-4b04-8eea-1706ae925403 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:54.663946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a0ce8-7e12-4b04-8eea-1706ae925403 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:54.666085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a0ce8-7e12-4b04-8eea-1706ae925403 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:54.682367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-768a0ce8-7e12-4b04-8eea-1706ae925403 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:54.682367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:54.682367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:54.682367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:54.682367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:54.682367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:54.682367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:54.682367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:54.682367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:54.682367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:54.682367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:54.682367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:54.682367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:14:54.682367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:14:54.682367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:14:54.683902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:14:54.683902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:54.683902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:54.683902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:54.683902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:54.683902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:54.683902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:54.683902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:54.683902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:54.683902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:54.683902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:54.683902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:54.683902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:54.683902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:54.683902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:54.683902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:54.683902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:54.684998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:14:54.684998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:14:54.684998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:14:54.684998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:14:54.684998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:14:54.684998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:14:54.684998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:54.684998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:54.684998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 348/698] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:54.684998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a0ce8-7e12-4b04-8eea-1706ae925403 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:54.712857 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ffccf-d19f-4498-b7e4-77c54acdcc3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:54.715271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ffccf-d19f-4498-b7e4-77c54acdcc3a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:54.747230 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:54.784841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:54.794014 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a0ce8-7e12-4b04-8eea-1706ae925403 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:54.830419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231947d8-f399-4ab7-b6e3-de6cd324f37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:54.831145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231947d8-f399-4ab7-b6e3-de6cd324f37c None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:54.831427 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231947d8-f399-4ab7-b6e3-de6cd324f37c None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:54.831697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231947d8-f399-4ab7-b6e3-de6cd324f37c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:54.842510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231947d8-f399-4ab7-b6e3-de6cd324f37c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:54.844325 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231947d8-f399-4ab7-b6e3-de6cd324f37c None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:54.859818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-231947d8-f399-4ab7-b6e3-de6cd324f37c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:54.859818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:54.859818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:54.859818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:54.859818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:54.859818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:54.859818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:54.859818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:54.859818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:54.859818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:54.859818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:54.859818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:54.859818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:54.859818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:54.859818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:54.859818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:54.860844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:54.860844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:54.860844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:54.860844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:54.860844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:54.860844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:54.860844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:54.860844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:54.860844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:54.860844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:54.860844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:54.860844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:54.860844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:54.860844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:54.860844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:54.860844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:54.860844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:54.861743 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:54.861743 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:54.861743 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:54.867610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 349/700] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:54.867610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231947d8-f399-4ab7-b6e3-de6cd324f37c None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:54.868695 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 351/700] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:14:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 14:14:54.872531 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ffccf-d19f-4498-b7e4-77c54acdcc3a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:54.973731 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231947d8-f399-4ab7-b6e3-de6cd324f37c None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:54.984200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:55.008590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ffccf-d19f-4498-b7e4-77c54acdcc3a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:55.010925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cc0fb-907e-41b5-88e7-515fcb0e237c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:55.011792 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cc0fb-907e-41b5-88e7-515fcb0e237c None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:55.012073 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cc0fb-907e-41b5-88e7-515fcb0e237c None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:55.012345 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cc0fb-907e-41b5-88e7-515fcb0e237c None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:55.013030 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cc0fb-907e-41b5-88e7-515fcb0e237c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:55.013331 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cc0fb-907e-41b5-88e7-515fcb0e237c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:55.014911 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cc0fb-907e-41b5-88e7-515fcb0e237c None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:55.028359 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 350/701] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:14:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:55.028959 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cc0fb-907e-41b5-88e7-515fcb0e237c None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:55.051209 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e84a65-c2d4-4b8e-9f9b-4319118547fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:55.051854 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e84a65-c2d4-4b8e-9f9b-4319118547fd None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:55.052118 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e84a65-c2d4-4b8e-9f9b-4319118547fd None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:55.052388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e84a65-c2d4-4b8e-9f9b-4319118547fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:55.063745 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e84a65-c2d4-4b8e-9f9b-4319118547fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:55.065183 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e84a65-c2d4-4b8e-9f9b-4319118547fd None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:55.077200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:55.084748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-42e84a65-c2d4-4b8e-9f9b-4319118547fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:55.084748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:55.084748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:55.084748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:55.084748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:55.084748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:55.084748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:55.084748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:55.084748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:55.084748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:55.084748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:55.084748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:55.084748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:55.084748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:55.084748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:55.084748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:55.092281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:55.092281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:55.092281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:55.092281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:55.092281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:55.092281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:55.092281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:55.092281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:55.092281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:55.092281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:55.092281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:55.092281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:55.092281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:55.092281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:55.092281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:55.092281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:55.092281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:55.093182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:55.093182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:55.093182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:55.093182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 352/702] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:55.093182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5abb24-1c81-485f-939e-2c555310181f None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:55.093182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5abb24-1c81-485f-939e-2c555310181f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:55.093182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5abb24-1c81-485f-939e-2c555310181f None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:55.093182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 353/703] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:55.093182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5abb24-1c81-485f-939e-2c555310181f None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:55.145689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cc0fb-907e-41b5-88e7-515fcb0e237c None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:55.183760 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bce138-2ce0-47cb-92ae-c88c62b33a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:55.184429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bce138-2ce0-47cb-92ae-c88c62b33a62 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:55.184643 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bce138-2ce0-47cb-92ae-c88c62b33a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:55.184912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bce138-2ce0-47cb-92ae-c88c62b33a62 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:55.185544 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bce138-2ce0-47cb-92ae-c88c62b33a62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:55.185843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bce138-2ce0-47cb-92ae-c88c62b33a62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:55.187256 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bce138-2ce0-47cb-92ae-c88c62b33a62 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:55.196780 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5abb24-1c81-485f-939e-2c555310181f None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:55.199635 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 351/704] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:55.200292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f855952-fe84-4e45-94cb-5be054ea50ec None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:55.200580 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f855952-fe84-4e45-94cb-5be054ea50ec None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:55.200852 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f855952-fe84-4e45-94cb-5be054ea50ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:55.218861 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-6f855952-fe84-4e45-94cb-5be054ea50ec None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:55.220032 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-6f855952-fe84-4e45-94cb-5be054ea50ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:55.237325 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42723c1-95cc-4f4f-833b-a3406b037d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:55.237876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42723c1-95cc-4f4f-833b-a3406b037d0e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:55.238189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42723c1-95cc-4f4f-833b-a3406b037d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:55.238496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42723c1-95cc-4f4f-833b-a3406b037d0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:55.239293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42723c1-95cc-4f4f-833b-a3406b037d0e None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=709b9ae82dcd47408fc9a6d54a54428a) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:55.239620 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42723c1-95cc-4f4f-833b-a3406b037d0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:55.241074 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42723c1-95cc-4f4f-833b-a3406b037d0e None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:55.245018 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:55.248450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 354/705] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:14:55 2022] GET /identity/v3/role_assignments?user.id=709b9ae82dcd47408fc9a6d54a54428a&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:55.249306 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42723c1-95cc-4f4f-833b-a3406b037d0e None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:55.314968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 352/706] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:55.316554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f855952-fe84-4e45-94cb-5be054ea50ec None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:55.339984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:55.389387 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42723c1-95cc-4f4f-833b-a3406b037d0e None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:55.440158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a10ebe-4166-482f-b6da-3f7307675358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:55.441289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a10ebe-4166-482f-b6da-3f7307675358 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:55.441981 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a10ebe-4166-482f-b6da-3f7307675358 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:55.442267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a10ebe-4166-482f-b6da-3f7307675358 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:55.443203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a10ebe-4166-482f-b6da-3f7307675358 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:55.443512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a10ebe-4166-482f-b6da-3f7307675358 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:55.444658 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f855952-fe84-4e45-94cb-5be054ea50ec None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:55.445002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a10ebe-4166-482f-b6da-3f7307675358 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:55.462970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 355/707] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:55.464274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22856c88-f599-4ecb-a066-f4c662ce2ac4 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:55.464557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22856c88-f599-4ecb-a066-f4c662ce2ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:55.464825 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22856c88-f599-4ecb-a066-f4c662ce2ac4 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:55.467067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 356/708] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:55.468647 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22856c88-f599-4ecb-a066-f4c662ce2ac4 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:55.494926 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c56d1-30f0-4ec8-84b1-023684904532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:55.495625 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c56d1-30f0-4ec8-84b1-023684904532 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:55.496220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c56d1-30f0-4ec8-84b1-023684904532 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:55.496514 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c56d1-30f0-4ec8-84b1-023684904532 None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:55.497158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c56d1-30f0-4ec8-84b1-023684904532 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:55.498653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c56d1-30f0-4ec8-84b1-023684904532 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:55.581348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 353/709] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:14:55 2022] POST /identity/v3/users => generated 254 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:14:55.582587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085f917-9256-4e65-a43c-ea96c2560d24 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:55.582899 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085f917-9256-4e65-a43c-ea96c2560d24 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:55.583176 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085f917-9256-4e65-a43c-ea96c2560d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:55.614477 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-c085f917-9256-4e65-a43c-ea96c2560d24 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:55.615740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c085f917-9256-4e65-a43c-ea96c2560d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:55.620979 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22856c88-f599-4ecb-a066-f4c662ce2ac4 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:55.630818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:55.650273 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:55.662158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31907a-aca2-41d5-b9dc-7e738956e6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:55.662851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31907a-aca2-41d5-b9dc-7e738956e6cc None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:55.663197 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31907a-aca2-41d5-b9dc-7e738956e6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:55.663475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31907a-aca2-41d5-b9dc-7e738956e6cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:55.672362 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31907a-aca2-41d5-b9dc-7e738956e6cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:55.673959 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31907a-aca2-41d5-b9dc-7e738956e6cc None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:55.686715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-1b31907a-aca2-41d5-b9dc-7e738956e6cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:55.686715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:55.686715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:55.686715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:55.686715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:55.686715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:55.686715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:55.686715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:55.686715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:55.686715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:55.686715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:55.686715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:55.686715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:14:55.686715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:14:55.686715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:14:55.687733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:14:55.687733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:55.687733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:55.687733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:55.687733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:55.687733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:55.687733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:55.687733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:55.687733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:55.687733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:55.687733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:55.687733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:55.687733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:55.687733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:55.687733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:55.687733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:55.687733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:55.688697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:14:55.688697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:14:55.688697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:14:55.688697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:14:55.688697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:14:55.688697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:14:55.688697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:14:55.688697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:55.688697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 357/710] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:55.689239 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31907a-aca2-41d5-b9dc-7e738956e6cc None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:55.723155 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 354/711] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:55.725071 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085f917-9256-4e65-a43c-ea96c2560d24 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:55.832204 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31907a-aca2-41d5-b9dc-7e738956e6cc None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:55.854404 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085f917-9256-4e65-a43c-ea96c2560d24 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:55.879173 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7dc32-3d3a-4180-81a6-1add754b5278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:55.879524 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7dc32-3d3a-4180-81a6-1add754b5278 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:55.879709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7dc32-3d3a-4180-81a6-1add754b5278 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:55.879984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7dc32-3d3a-4180-81a6-1add754b5278 None admin] PATH_INFO: `/v3/services/network` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:55.880727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7dc32-3d3a-4180-81a6-1add754b5278 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:55.882159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7dc32-3d3a-4180-81a6-1add754b5278 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:55.894945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-02c7dc32-3d3a-4180-81a6-1add754b5278 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 14:14:55.894945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:55.894945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:14:55.894945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:14:55.894945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:14:55.894945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:14:55.894945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:14:55.894945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:14:55.894945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 14:14:55.894945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:55.894945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:14:55.894945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:55.894945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:55.894945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:55.894945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:55.894945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:55.894945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:55.902130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:55.902130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:55.902130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:55.902130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:55.902130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:55.902130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:55.902130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:14:55.902130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:14:55.902130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:14:55.902130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:14:55.902130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:55.902130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:55.902130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:55.902130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:55.902130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:55.902130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:55.903075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:55.903075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:55.903075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:55.903075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:55.903075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:55.903075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:55.903075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:55.903075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:55.903075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:55.903075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:55.903075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:14:55.903075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:14:55.903075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 14:14:55.903075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:55.903075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 358/712] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:14:55 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:55.904010 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7dc32-3d3a-4180-81a6-1add754b5278 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:55.906324 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8239a9-3359-434a-b8bc-ba45a53b887e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:55.907260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8239a9-3359-434a-b8bc-ba45a53b887e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:55.907806 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8239a9-3359-434a-b8bc-ba45a53b887e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:55.908298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8239a9-3359-434a-b8bc-ba45a53b887e None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:55.909217 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8239a9-3359-434a-b8bc-ba45a53b887e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:55.909762 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8239a9-3359-434a-b8bc-ba45a53b887e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:55.911448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8239a9-3359-434a-b8bc-ba45a53b887e None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:55.925593 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 355/713] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:14:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:55.931157 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8239a9-3359-434a-b8bc-ba45a53b887e None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:55.935897 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:55.969128 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:56.037073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7dc32-3d3a-4180-81a6-1add754b5278 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:56.058312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8239a9-3359-434a-b8bc-ba45a53b887e None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:56.081607 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597c562-68b9-46d7-bb47-eebf2b7de050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:56.082287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597c562-68b9-46d7-bb47-eebf2b7de050 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:56.082517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597c562-68b9-46d7-bb47-eebf2b7de050 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:56.082918 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597c562-68b9-46d7-bb47-eebf2b7de050 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:56.083563 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f597c562-68b9-46d7-bb47-eebf2b7de050 None admin] RBAC: Adding query filter params (name=network) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:56.083972 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f597c562-68b9-46d7-bb47-eebf2b7de050 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:56.085356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f597c562-68b9-46d7-bb47-eebf2b7de050 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:56.090937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 359/714] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 5 14:14:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:56.091755 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597c562-68b9-46d7-bb47-eebf2b7de050 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:56.111362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0f977-3cb9-4d17-aa2f-9e0bac1abc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:56.111973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0f977-3cb9-4d17-aa2f-9e0bac1abc08 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:56.112280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0f977-3cb9-4d17-aa2f-9e0bac1abc08 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:56.112625 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0f977-3cb9-4d17-aa2f-9e0bac1abc08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:56.113452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b0f977-3cb9-4d17-aa2f-9e0bac1abc08 None admin] RBAC: Adding query filter params (role.id=0a83e40e2d64470a8489bf548e34dabd, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=0a85809d14cb449d848f22a3f39bf641) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:56.113768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b0f977-3cb9-4d17-aa2f-9e0bac1abc08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:56.115219 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b0f977-3cb9-4d17-aa2f-9e0bac1abc08 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:56.122178 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 356/715] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:14:55 2022] GET /identity/v3/role_assignments?user.id=0a85809d14cb449d848f22a3f39bf641&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=0a83e40e2d64470a8489bf548e34dabd => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:56.131392 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0f977-3cb9-4d17-aa2f-9e0bac1abc08 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:56.216079 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597c562-68b9-46d7-bb47-eebf2b7de050 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:56.218466 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:56.249414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:56.257381 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0f977-3cb9-4d17-aa2f-9e0bac1abc08 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:56.296957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b175b5-900f-41ba-93f4-b48943ca1a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:56.297585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b175b5-900f-41ba-93f4-b48943ca1a5d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:56.297878 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b175b5-900f-41ba-93f4-b48943ca1a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:56.298161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b175b5-900f-41ba-93f4-b48943ca1a5d None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:56.298816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b175b5-900f-41ba-93f4-b48943ca1a5d None admin] RBAC: Adding query filter params (type=network) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:56.299315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b175b5-900f-41ba-93f4-b48943ca1a5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:56.300752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b175b5-900f-41ba-93f4-b48943ca1a5d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:56.303651 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550f48f-4f9b-4e39-b0a0-39c206a26821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:56.304320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550f48f-4f9b-4e39-b0a0-39c206a26821 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:56.304604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550f48f-4f9b-4e39-b0a0-39c206a26821 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:56.304882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550f48f-4f9b-4e39-b0a0-39c206a26821 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:56.305480 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550f48f-4f9b-4e39-b0a0-39c206a26821 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:56.307457 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550f48f-4f9b-4e39-b0a0-39c206a26821 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:56.308256 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 357/716] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 5 14:14:56 2022] GET /identity/v3/services?type=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:56.333140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-0550f48f-4f9b-4e39-b0a0-39c206a26821 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 14:14:56.333140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:56.333140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:14:56.333140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:14:56.333140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:14:56.333140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:14:56.333140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:14:56.333140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:14:56.333140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 14:14:56.333140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:56.333140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:14:56.333140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:56.333140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:56.333140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:56.333140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:56.333140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:56.333140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:56.334193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:56.334193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:56.334193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:56.334193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:56.334193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:56.334193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:56.334193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:14:56.334193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:14:56.334193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:14:56.334193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:14:56.334193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:56.334193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:56.334193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:56.334193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:56.334193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:56.334193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:56.335732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:56.335732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:56.335732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:56.335732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:56.335732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:56.335732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:56.335732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:56.335732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:56.335732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:56.335732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:56.335732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:14:56.335732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:14:56.335732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 14:14:56.335732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:56.335732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 360/717] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Sep 5 14:14:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:56.340450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b175b5-900f-41ba-93f4-b48943ca1a5d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:56.509353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b175b5-900f-41ba-93f4-b48943ca1a5d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:56.545461 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:56.550298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03719ba0-b698-4b89-b166-b7c6601c1198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:56.550935 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03719ba0-b698-4b89-b166-b7c6601c1198 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:56.551212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03719ba0-b698-4b89-b166-b7c6601c1198 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:56.551481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03719ba0-b698-4b89-b166-b7c6601c1198 None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:56.552100 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03719ba0-b698-4b89-b166-b7c6601c1198 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:56.552395 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03719ba0-b698-4b89-b166-b7c6601c1198 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:56.553762 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03719ba0-b698-4b89-b166-b7c6601c1198 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:56.559053 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 358/718] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Sep 5 14:14:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:56.564820 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550f48f-4f9b-4e39-b0a0-39c206a26821 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:56.674362 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550f48f-4f9b-4e39-b0a0-39c206a26821 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:56.709868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fed011-b1ad-40f3-a191-686ea09db664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:56.710475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fed011-b1ad-40f3-a191-686ea09db664 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:56.710751 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fed011-b1ad-40f3-a191-686ea09db664 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:56.711063 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fed011-b1ad-40f3-a191-686ea09db664 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:56.711687 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fed011-b1ad-40f3-a191-686ea09db664 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:56.712135 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fed011-b1ad-40f3-a191-686ea09db664 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:56.713570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fed011-b1ad-40f3-a191-686ea09db664 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:56.719595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 361/719] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Sep 5 14:14:56 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:57.365941 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f053e-9260-44d0-9e97-935a58b7b52f None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:57.366448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f053e-9260-44d0-9e97-935a58b7b52f None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:57.366448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f053e-9260-44d0-9e97-935a58b7b52f None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:57.367768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 359/720] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:57.373750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848eb7b8-89b2-407c-b583-a987334c2237 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:57.374291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848eb7b8-89b2-407c-b583-a987334c2237 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:57.374821 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848eb7b8-89b2-407c-b583-a987334c2237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:57.393932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-848eb7b8-89b2-407c-b583-a987334c2237 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:57.399584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-848eb7b8-89b2-407c-b583-a987334c2237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:57.436796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:57.472973 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 362/721] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:57.864242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f053e-9260-44d0-9e97-935a58b7b52f None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:57.966662 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a6f16-d101-4d57-9bf0-1bb3214bf817 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:57.967233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a6f16-d101-4d57-9bf0-1bb3214bf817 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:57.967233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a6f16-d101-4d57-9bf0-1bb3214bf817 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:57.968790 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 363/722] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:14:57.976150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f053e-9260-44d0-9e97-935a58b7b52f None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:57.980587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e2731-7298-4c1b-91e3-3ffdfa2efe6e None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:57.980876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e2731-7298-4c1b-91e3-3ffdfa2efe6e None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:57.981148 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e2731-7298-4c1b-91e3-3ffdfa2efe6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:58.012865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-b72e2731-7298-4c1b-91e3-3ffdfa2efe6e None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:58.019519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b72e2731-7298-4c1b-91e3-3ffdfa2efe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:58.022075 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93566593-b691-4381-be0d-d69e9f649202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:58.022713 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93566593-b691-4381-be0d-d69e9f649202 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:58.023040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93566593-b691-4381-be0d-d69e9f649202 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:58.023313 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93566593-b691-4381-be0d-d69e9f649202 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:58.042753 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93566593-b691-4381-be0d-d69e9f649202 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:58.048776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-93566593-b691-4381-be0d-d69e9f649202 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:14:58.048776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:58.048776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:58.048776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:58.048776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:58.048776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:58.048776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:58.048776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:58.048776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:58.048776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:58.048776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:58.048776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:58.048776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:58.048776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:58.048776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:58.048776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:58.049758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:58.049758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:58.049758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:58.049758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:58.049758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:58.049758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:58.049758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:58.049758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:58.049758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:58.049758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:58.049758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:58.049758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:58.049758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:58.049758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:58.049758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:58.049758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:58.050632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:58.050632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:58.050632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:58.050632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:58.050632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:58.050632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:58.050632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:58.050632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:58.050632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:58.050632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:58.050632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:14:58.050632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:58.051917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 360/723] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:14:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:58.058309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93566593-b691-4381-be0d-d69e9f649202 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:58.160654 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:58.161457 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 364/724] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:58.226594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93566593-b691-4381-be0d-d69e9f649202 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:58.273230 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397aa9ba-fc8c-457b-83d9-aada0069eac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:58.273891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397aa9ba-fc8c-457b-83d9-aada0069eac0 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:58.274180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397aa9ba-fc8c-457b-83d9-aada0069eac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:58.274600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397aa9ba-fc8c-457b-83d9-aada0069eac0 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:58.275219 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397aa9ba-fc8c-457b-83d9-aada0069eac0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:58.275526 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397aa9ba-fc8c-457b-83d9-aada0069eac0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:58.276911 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397aa9ba-fc8c-457b-83d9-aada0069eac0 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:58.286589 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 361/725] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:14:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:14:58.292367 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e2731-7298-4c1b-91e3-3ffdfa2efe6e None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:58.403198 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:58.411439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e2731-7298-4c1b-91e3-3ffdfa2efe6e None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:58.462732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980954f-53c0-4746-af7a-3c57c828eac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:58.463736 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980954f-53c0-4746-af7a-3c57c828eac4 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:58.464255 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980954f-53c0-4746-af7a-3c57c828eac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:58.464721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980954f-53c0-4746-af7a-3c57c828eac4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:58.479779 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980954f-53c0-4746-af7a-3c57c828eac4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:58.481305 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980954f-53c0-4746-af7a-3c57c828eac4 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:58.487989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397aa9ba-fc8c-457b-83d9-aada0069eac0 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:58.509865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-4980954f-53c0-4746-af7a-3c57c828eac4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:58.509865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:58.509865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:58.509865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:58.509865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:58.509865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:58.509865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:58.509865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:58.509865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:58.509865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:58.509865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:58.509865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:58.509865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:58.509865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:58.509865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:58.509865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:58.510965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:58.510965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:58.510965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:58.510965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:58.510965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:58.510965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:58.510965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:58.510965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:58.510965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:58.510965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:58.510965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:58.510965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:58.510965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:58.510965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:58.510965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:58.510965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:58.510965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:58.511904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:58.511904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:58.511904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:58.514657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 365/726] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:58.517542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980954f-53c0-4746-af7a-3c57c828eac4 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:58.679126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980954f-53c0-4746-af7a-3c57c828eac4 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:58.683082 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397aa9ba-fc8c-457b-83d9-aada0069eac0 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:58.715492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:58.747026 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eba1ba-16bc-49ca-96f9-fba5cab100bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:58.747698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eba1ba-16bc-49ca-96f9-fba5cab100bb None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:58.747996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eba1ba-16bc-49ca-96f9-fba5cab100bb None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:58.748246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eba1ba-16bc-49ca-96f9-fba5cab100bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:58.750270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda144b-0aca-43ab-ad7a-e12bb936974d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:58.750904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda144b-0aca-43ab-ad7a-e12bb936974d None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:58.751186 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda144b-0aca-43ab-ad7a-e12bb936974d None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:58.751495 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda144b-0aca-43ab-ad7a-e12bb936974d None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:58.752224 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda144b-0aca-43ab-ad7a-e12bb936974d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:58.752671 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda144b-0aca-43ab-ad7a-e12bb936974d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:58.753906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda144b-0aca-43ab-ad7a-e12bb936974d None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:58.772937 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eba1ba-16bc-49ca-96f9-fba5cab100bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:58.780181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-92eba1ba-16bc-49ca-96f9-fba5cab100bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:58.780181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:58.780181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:58.780181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:58.780181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:58.780181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:58.780181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:58.780181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:58.780181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:58.780181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:58.780181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:58.780181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:58.780181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:58.780181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:58.780181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:58.780181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:58.781141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:58.781141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:58.781141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:58.781141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:58.781141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:58.781141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:58.781141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:58.781141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:58.781141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:58.781141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:58.781141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:58.781141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:58.781141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:58.781141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:58.781141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:58.781141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:58.781996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:58.781996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:58.781996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:58.781996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:58.781996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:58.781996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:58.781996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:58.781996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:58.781996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:58.781996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:58.781996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:14:58.781996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:58.781996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 362/727] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:14:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:14:58.786693 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 366/728] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:58.787594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eba1ba-16bc-49ca-96f9-fba5cab100bb None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:58.795368 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda144b-0aca-43ab-ad7a-e12bb936974d None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:58.822802 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:58.910920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eba1ba-16bc-49ca-96f9-fba5cab100bb None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:58.915792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda144b-0aca-43ab-ad7a-e12bb936974d None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:58.948363 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea464b-c184-4812-b77c-35c99441da39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:58.948900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea464b-c184-4812-b77c-35c99441da39 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:58.949179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea464b-c184-4812-b77c-35c99441da39 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:58.949630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea464b-c184-4812-b77c-35c99441da39 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:58.950308 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea464b-c184-4812-b77c-35c99441da39 None admin] RBAC: Adding query filter params (name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:58.950618 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea464b-c184-4812-b77c-35c99441da39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:58.952027 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea464b-c184-4812-b77c-35c99441da39 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:58.953117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3159f-603d-40d6-9b65-10e21cb935d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:58.953743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3159f-603d-40d6-9b65-10e21cb935d2 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:58.954022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3159f-603d-40d6-9b65-10e21cb935d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:58.954464 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3159f-603d-40d6-9b65-10e21cb935d2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:58.960433 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 363/729] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:14:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:58.961695 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3159f-603d-40d6-9b65-10e21cb935d2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:58.963163 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3159f-603d-40d6-9b65-10e21cb935d2 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:58.965810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113c88e-72ed-4efb-95de-1f6f2535897c None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:58.966104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113c88e-72ed-4efb-95de-1f6f2535897c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:58.966387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113c88e-72ed-4efb-95de-1f6f2535897c None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:58.971776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 364/730] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:58.971776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5c3ca-23a7-488c-b668-98aec61c3799 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:58.971776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5c3ca-23a7-488c-b668-98aec61c3799 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:58.971776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5c3ca-23a7-488c-b668-98aec61c3799 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:58.971776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 365/731] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:58.971776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5c3ca-23a7-488c-b668-98aec61c3799 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:58.978481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-2aa3159f-603d-40d6-9b65-10e21cb935d2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:14:58.978481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:58.978481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:58.978481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:58.978481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:58.978481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:58.978481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:58.978481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:58.978481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:58.978481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:58.978481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:58.978481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:58.978481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:58.978481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:58.978481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:58.978481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:58.979781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:58.980740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:58.980740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:58.980740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:58.980740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:58.980740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:58.980740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:58.980740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:58.980740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:58.980740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:14:58.980740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:58.980740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 367/732] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 5 14:14:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:58.981333 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e99951-e154-4428-bccd-afa2056bab6e None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:58.981333 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e99951-e154-4428-bccd-afa2056bab6e None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:58.981333 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e99951-e154-4428-bccd-afa2056bab6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:58.999877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-74e99951-e154-4428-bccd-afa2056bab6e None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:59.001011 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-74e99951-e154-4428-bccd-afa2056bab6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:59.025889 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:59.077986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 368/733] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:59.077986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167782aa-7eba-46e5-9f18-a2c1cc03f4a3 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:59.077986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167782aa-7eba-46e5-9f18-a2c1cc03f4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:59.077986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167782aa-7eba-46e5-9f18-a2c1cc03f4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:59.082022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:59.101348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-167782aa-7eba-46e5-9f18-a2c1cc03f4a3 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:59.103176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-167782aa-7eba-46e5-9f18-a2c1cc03f4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:59.182371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 369/734] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:59.183575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167782aa-7eba-46e5-9f18-a2c1cc03f4a3 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:59.248069 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5c3ca-23a7-488c-b668-98aec61c3799 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:59.282087 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7185c-6369-41bb-a9b8-02d3aeb3dd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:59.282711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7185c-6369-41bb-a9b8-02d3aeb3dd7b None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:59.283036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7185c-6369-41bb-a9b8-02d3aeb3dd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:59.283367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7185c-6369-41bb-a9b8-02d3aeb3dd7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:59.293129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7185c-6369-41bb-a9b8-02d3aeb3dd7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:59.294578 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7185c-6369-41bb-a9b8-02d3aeb3dd7b None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:59.306934 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167782aa-7eba-46e5-9f18-a2c1cc03f4a3 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:59.310017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-19c7185c-6369-41bb-a9b8-02d3aeb3dd7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:59.310017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:59.310017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:59.310017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:59.310017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:59.310017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:59.310017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:59.310017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:59.310017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:59.310017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:59.310017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:59.310017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:59.310017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:59.310017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:59.310017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:59.310017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:59.311265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:59.311265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:59.311265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:59.311265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:59.311265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:59.311265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:59.311265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:59.311265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:59.311265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:59.311265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:59.311265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:59.311265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:59.311265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:59.311265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:59.311265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:59.311265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:59.311265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:59.312204 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:59.312204 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:59.312204 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:59.312204 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 366/735] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:59.312204 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d709e5-985b-4c72-a91c-4ca5b06ea5a2 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:59.312582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d709e5-985b-4c72-a91c-4ca5b06ea5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:59.312658 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d709e5-985b-4c72-a91c-4ca5b06ea5a2 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:59.318199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 367/736] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:14:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:14:59.318996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d709e5-985b-4c72-a91c-4ca5b06ea5a2 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:59.346333 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:59.351288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9013cd-4827-4d99-bb99-fcf8cb627cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:59.351902 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9013cd-4827-4d99-bb99-fcf8cb627cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:59.352209 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9013cd-4827-4d99-bb99-fcf8cb627cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:59.352509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9013cd-4827-4d99-bb99-fcf8cb627cf9 None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:59.353249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9013cd-4827-4d99-bb99-fcf8cb627cf9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:59.353559 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9013cd-4827-4d99-bb99-fcf8cb627cf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:59.355012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9013cd-4827-4d99-bb99-fcf8cb627cf9 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:59.374031 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 370/737] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 5 14:14:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:59.375354 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135b68d-3353-48a5-b904-f01f4b33a984 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:59.375642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135b68d-3353-48a5-b904-f01f4b33a984 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:59.375916 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135b68d-3353-48a5-b904-f01f4b33a984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:59.394076 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-8135b68d-3353-48a5-b904-f01f4b33a984 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:14:59.395300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8135b68d-3353-48a5-b904-f01f4b33a984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:14:59.424184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:59.433715 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d709e5-985b-4c72-a91c-4ca5b06ea5a2 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:59.467165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 371/738] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 14:14:59.467873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135b68d-3353-48a5-b904-f01f4b33a984 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:59.485145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7e04a-39f9-4881-8a44-28562f8023fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:59.485849 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7e04a-39f9-4881-8a44-28562f8023fb None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:59.486134 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7e04a-39f9-4881-8a44-28562f8023fb None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:59.486406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7e04a-39f9-4881-8a44-28562f8023fb None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:59.487096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7e04a-39f9-4881-8a44-28562f8023fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:14:59.487398 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7e04a-39f9-4881-8a44-28562f8023fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:59.488904 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7e04a-39f9-4881-8a44-28562f8023fb None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:59.503001 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 368/739] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:14:59.503216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7e04a-39f9-4881-8a44-28562f8023fb None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:59.574273 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135b68d-3353-48a5-b904-f01f4b33a984 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:59.597455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:59.608467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7e04a-39f9-4881-8a44-28562f8023fb None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:59.612769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba86b8-1814-4e9c-9f09-8b9293149eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:59.613383 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba86b8-1814-4e9c-9f09-8b9293149eb4 None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:59.613667 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba86b8-1814-4e9c-9f09-8b9293149eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:59.613943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba86b8-1814-4e9c-9f09-8b9293149eb4 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:59.614618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba86b8-1814-4e9c-9f09-8b9293149eb4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:59.616019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba86b8-1814-4e9c-9f09-8b9293149eb4 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:59.642526 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5d5f4-0ce0-4978-8b39-88d27a99dee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:59.643225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5d5f4-0ce0-4978-8b39-88d27a99dee2 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:59.643503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5d5f4-0ce0-4978-8b39-88d27a99dee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:59.643772 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5d5f4-0ce0-4978-8b39-88d27a99dee2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:59.653979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5d5f4-0ce0-4978-8b39-88d27a99dee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:59.655443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5d5f4-0ce0-4978-8b39-88d27a99dee2 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:59.669439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-34e5d5f4-0ce0-4978-8b39-88d27a99dee2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:59.669439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:59.669439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:59.669439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:59.669439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:59.669439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:59.669439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:59.669439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:59.669439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:59.669439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:59.669439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:59.669439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:59.669439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:14:59.669439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:14:59.669439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:14:59.669439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:14:59.670577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:59.670577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:59.670577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:59.670577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:59.670577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:59.670577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:59.670577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:59.670577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:59.670577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:59.670577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:59.670577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:59.670577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:59.670577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:59.670577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:59.670577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:59.670577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:59.670577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:14:59.671584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:14:59.671584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:14:59.671584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:59.671584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 369/740] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:14:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:59.671584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5d5f4-0ce0-4978-8b39-88d27a99dee2 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:59.678179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:59.730841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 372/741] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 5 14:14:59 2022] POST /identity/v3/services => generated 240 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:14:59.730841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba86b8-1814-4e9c-9f09-8b9293149eb4 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:59.796006 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5d5f4-0ce0-4978-8b39-88d27a99dee2 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:59.839254 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71319936-1190-4e29-9440-ff697a2b9083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:59.839897 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71319936-1190-4e29-9440-ff697a2b9083 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:59.840182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71319936-1190-4e29-9440-ff697a2b9083 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:59.840454 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71319936-1190-4e29-9440-ff697a2b9083 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:59.846769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba86b8-1814-4e9c-9f09-8b9293149eb4 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:14:59.851550 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71319936-1190-4e29-9440-ff697a2b9083 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:14:59.856985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-71319936-1190-4e29-9440-ff697a2b9083 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:14:59.856985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:59.856985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:59.856985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:59.856985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:59.856985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:59.856985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:59.856985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:59.856985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:59.856985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:59.856985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:59.856985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:59.856985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:14:59.856985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:14:59.856985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:14:59.856985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:14:59.857969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:14:59.857969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:14:59.857969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:14:59.857969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:14:59.857969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:59.857969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:59.857969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:59.857969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:59.857969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:59.857969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:59.857969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:59.857969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:59.857969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:59.857969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:59.857969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:59.857969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:59.859136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:59.859136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:59.859136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:59.859136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:59.859136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:14:59.859136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:14:59.859136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:14:59.859136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:14:59.859136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:14:59.859136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:14:59.859136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:14:59.859136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:14:59.859136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 370/742] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:14:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:59.859136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71319936-1190-4e29-9440-ff697a2b9083 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:59.895694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1b1e2-d704-4469-9a39-43ec26431f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:14:59.896345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b1e2-d704-4469-9a39-43ec26431f97 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:14:59.896626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b1e2-d704-4469-9a39-43ec26431f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:14:59.896903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b1e2-d704-4469-9a39-43ec26431f97 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:14:59.905651 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1b1e2-d704-4469-9a39-43ec26431f97 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:14:59.907144 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1b1e2-d704-4469-9a39-43ec26431f97 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:14:59.918445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-8cb1b1e2-d704-4469-9a39-43ec26431f97 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 14:14:59.918445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:14:59.918445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:14:59.918445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:14:59.918445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:14:59.918445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:14:59.918445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:14:59.918445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:14:59.918445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:14:59.918445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:14:59.918445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:14:59.918445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:14:59.918445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:14:59.918445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:14:59.918445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:14:59.918445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:14:59.919584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:14:59.920575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:14:59.920575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:14:59.920575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:14:59.920575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:14:59.920575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:14:59.920575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:14:59.920575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:14:59.920575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:14:59.920575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 14:14:59.920575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:14:59.920575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 373/743] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 5 14:14:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:14:59.921201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1b1e2-d704-4469-9a39-43ec26431f97 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:14:59.929696 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:59.965855 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:14:59.972692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71319936-1190-4e29-9440-ff697a2b9083 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:00.009108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f55a46-b9c9-4eca-88bf-0419756bf177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:00.009715 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f55a46-b9c9-4eca-88bf-0419756bf177 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:00.009996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f55a46-b9c9-4eca-88bf-0419756bf177 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:00.010278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f55a46-b9c9-4eca-88bf-0419756bf177 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:00.010940 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f55a46-b9c9-4eca-88bf-0419756bf177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:00.011427 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f55a46-b9c9-4eca-88bf-0419756bf177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:00.012608 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f55a46-b9c9-4eca-88bf-0419756bf177 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:00.026427 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 371/744] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:00.027307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f55a46-b9c9-4eca-88bf-0419756bf177 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:00.036575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1b1e2-d704-4469-9a39-43ec26431f97 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:00.075870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d47223-ee1a-4e4a-842d-5de769a813e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:00.075870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d47223-ee1a-4e4a-842d-5de769a813e2 None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:00.075870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d47223-ee1a-4e4a-842d-5de769a813e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:00.075870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d47223-ee1a-4e4a-842d-5de769a813e2 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:00.076298 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d47223-ee1a-4e4a-842d-5de769a813e2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:00.076298 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d47223-ee1a-4e4a-842d-5de769a813e2 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:00.137421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f55a46-b9c9-4eca-88bf-0419756bf177 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:00.153358 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 374/745] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 5 14:14:59 2022] POST /identity/v3/services => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:15:00.158107 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:00.159995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d47223-ee1a-4e4a-842d-5de769a813e2 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:00.185469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1fc8f-16aa-4ad1-a4ca-56f4876ff75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:00.186770 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1fc8f-16aa-4ad1-a4ca-56f4876ff75f None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:00.187509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1fc8f-16aa-4ad1-a4ca-56f4876ff75f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:00.187815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1fc8f-16aa-4ad1-a4ca-56f4876ff75f None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:00.188588 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1fc8f-16aa-4ad1-a4ca-56f4876ff75f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:00.188930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1fc8f-16aa-4ad1-a4ca-56f4876ff75f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:00.191182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1fc8f-16aa-4ad1-a4ca-56f4876ff75f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:00.204174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 372/746] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:15:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:00.209805 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1fc8f-16aa-4ad1-a4ca-56f4876ff75f None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:00.295071 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:00.302053 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d47223-ee1a-4e4a-842d-5de769a813e2 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:00.347408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1fc8f-16aa-4ad1-a4ca-56f4876ff75f None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:00.347588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f15aaa-31be-42f6-bbec-30709f294280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:00.347954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15aaa-31be-42f6-bbec-30709f294280 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:00.348239 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15aaa-31be-42f6-bbec-30709f294280 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:00.348567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15aaa-31be-42f6-bbec-30709f294280 None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:00.349306 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f15aaa-31be-42f6-bbec-30709f294280 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:00.349614 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f15aaa-31be-42f6-bbec-30709f294280 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:00.351114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f15aaa-31be-42f6-bbec-30709f294280 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:00.372847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 375/747] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Sep 5 14:15:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:00.372847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f15aaa-31be-42f6-bbec-30709f294280 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:00.400787 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f23a45-e59c-451f-9929-89297fefb86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:00.402337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f23a45-e59c-451f-9929-89297fefb86f None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:00.402778 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f23a45-e59c-451f-9929-89297fefb86f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:00.403551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f23a45-e59c-451f-9929-89297fefb86f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:00.413045 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f23a45-e59c-451f-9929-89297fefb86f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:00.415839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f23a45-e59c-451f-9929-89297fefb86f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:00.432202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-74f23a45-e59c-451f-9929-89297fefb86f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:00.432202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:00.432202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:00.432202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:00.432202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:00.432202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:00.432202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:00.432202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:00.432202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:00.432202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:00.432202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:00.432202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:00.432202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:15:00.432202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:15:00.432202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:15:00.433343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:15:00.433343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:00.433343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:00.433343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:00.433343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:00.433343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:00.433343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:00.433343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:00.433343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:00.433343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:00.433343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:00.433343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:00.433343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:00.433343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:00.433343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:00.433343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:00.433343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:00.434452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:15:00.434452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:15:00.434452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:15:00.434452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:15:00.434452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:15:00.434452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:15:00.434452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:00.434452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:00.437127 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 373/748] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:00.438440 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f23a45-e59c-451f-9929-89297fefb86f None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:00.484974 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f15aaa-31be-42f6-bbec-30709f294280 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:00.495482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:00.528259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4e863-e33e-493c-b005-e4f283351ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:00.530085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4e863-e33e-493c-b005-e4f283351ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:00.530521 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4e863-e33e-493c-b005-e4f283351ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:00.531572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4e863-e33e-493c-b005-e4f283351ad4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:00.546388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc4e863-e33e-493c-b005-e4f283351ad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:00.548895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc4e863-e33e-493c-b005-e4f283351ad4 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:00.566411 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f23a45-e59c-451f-9929-89297fefb86f None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:00.570311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-7fc4e863-e33e-493c-b005-e4f283351ad4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:00.570311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:00.570311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:00.570311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:00.570311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:00.570311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:00.570311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:00.570311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:00.570311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:00.570311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:00.570311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:00.570311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:00.570311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:00.570311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:00.570311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:00.570311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:00.572019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:00.572019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:00.572019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:00.572019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:00.572019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:00.572019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:00.572019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:00.572019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:00.572019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:00.572019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:00.572019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:00.572019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:00.572019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:00.572019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:00.572019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:00.572019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:00.572019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:00.573160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:00.573160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:00.573160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:00.574842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 376/749] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:00.575827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4e863-e33e-493c-b005-e4f283351ad4 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:00.592637 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:00.615957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb3235-0449-400b-8a79-2c9baa9a87a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:00.617122 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb3235-0449-400b-8a79-2c9baa9a87a7 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:00.617756 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb3235-0449-400b-8a79-2c9baa9a87a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:00.618243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb3235-0449-400b-8a79-2c9baa9a87a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:00.693029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4e863-e33e-493c-b005-e4f283351ad4 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:00.713842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb3235-0449-400b-8a79-2c9baa9a87a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:00.716382 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb3235-0449-400b-8a79-2c9baa9a87a7 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:00.730559 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e364d-034d-4f46-9d17-7bea7d1768e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:00.731121 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e364d-034d-4f46-9d17-7bea7d1768e1 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:00.731499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e364d-034d-4f46-9d17-7bea7d1768e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:00.731867 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e364d-034d-4f46-9d17-7bea7d1768e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:00.732857 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e364d-034d-4f46-9d17-7bea7d1768e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:00.733307 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e364d-034d-4f46-9d17-7bea7d1768e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:00.734883 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e364d-034d-4f46-9d17-7bea7d1768e1 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:00.753371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 377/750] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:15:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:00.754217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e364d-034d-4f46-9d17-7bea7d1768e1 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:00.768070 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:00.836358 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:00.846211 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 374/751] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:15:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:15:00.846211 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb3235-0449-400b-8a79-2c9baa9a87a7 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:00.872379 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e364d-034d-4f46-9d17-7bea7d1768e1 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:00.908521 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99c518-a54e-4513-a239-b687256da026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:00.909259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99c518-a54e-4513-a239-b687256da026 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:00.909664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99c518-a54e-4513-a239-b687256da026 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:00.910015 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99c518-a54e-4513-a239-b687256da026 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:00.910737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99c518-a54e-4513-a239-b687256da026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:00.911178 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99c518-a54e-4513-a239-b687256da026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:00.912619 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99c518-a54e-4513-a239-b687256da026 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:00.928712 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 378/752] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:00.928712 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99c518-a54e-4513-a239-b687256da026 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:00.998184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb3235-0449-400b-8a79-2c9baa9a87a7 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:01.043109 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f93999-2939-406a-b03e-5417f8de1495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:01.044016 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f93999-2939-406a-b03e-5417f8de1495 None admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:01.044440 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f93999-2939-406a-b03e-5417f8de1495 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:01.044826 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99c518-a54e-4513-a239-b687256da026 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:01.045346 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f93999-2939-406a-b03e-5417f8de1495 None admin] PATH_INFO: `/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/709b9ae82dcd47408fc9a6d54a54428a/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:01.048556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f93999-2939-406a-b03e-5417f8de1495 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9ca42deaed344eea726fe6e32356efc, user_id=709b9ae82dcd47408fc9a6d54a54428a, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:01.050143 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f93999-2939-406a-b03e-5417f8de1495 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:01.078529 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:01.084933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50e083-3a00-4236-b22e-5c9eb4898e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:01.085722 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50e083-3a00-4236-b22e-5c9eb4898e61 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:01.086072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50e083-3a00-4236-b22e-5c9eb4898e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:01.086480 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50e083-3a00-4236-b22e-5c9eb4898e61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:01.097489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50e083-3a00-4236-b22e-5c9eb4898e61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:01.099089 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50e083-3a00-4236-b22e-5c9eb4898e61 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:01.103177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:01.109123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 375/753] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:15:00 2022] PUT /identity/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/709b9ae82dcd47408fc9a6d54a54428a/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:15:01.110675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f93999-2939-406a-b03e-5417f8de1495 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:01.122960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-af50e083-3a00-4236-b22e-5c9eb4898e61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:01.122960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:01.122960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:01.122960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:01.122960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:01.122960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:01.122960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:01.122960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:01.122960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:01.122960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:01.122960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:01.122960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:01.122960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:01.122960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:01.122960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:01.122960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:01.123954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:01.123954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:01.123954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:01.123954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:01.123954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:01.123954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:01.123954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:01.123954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:01.123954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:01.123954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:01.123954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:01.123954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:01.123954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:01.123954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:01.123954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:01.123954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:01.123954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:01.124850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:01.124850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:01.124850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:01.128678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 379/754] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:01.130656 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50e083-3a00-4236-b22e-5c9eb4898e61 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:01.239977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f93999-2939-406a-b03e-5417f8de1495 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:01.280989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a1fe2-8d8b-4bf6-ad4d-17c26bf9fb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:01.282033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a1fe2-8d8b-4bf6-ad4d-17c26bf9fb76 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:01.282528 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a1fe2-8d8b-4bf6-ad4d-17c26bf9fb76 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:01.283035 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a1fe2-8d8b-4bf6-ad4d-17c26bf9fb76 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:01.292963 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a1fe2-8d8b-4bf6-ad4d-17c26bf9fb76 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:01.294869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a1fe2-8d8b-4bf6-ad4d-17c26bf9fb76 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:01.309383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-7c3a1fe2-8d8b-4bf6-ad4d-17c26bf9fb76 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 14:15:01.309383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:01.309383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:01.309383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:01.309383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:01.309383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:01.309383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:01.309383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:01.309383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:01.309383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:01.309383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:01.309383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:01.309383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:15:01.309383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:15:01.309383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:15:01.309383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:01.310413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:01.311482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:01.311482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:01.311482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:15:01.311482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:15:01.311482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:15:01.311482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:15:01.311482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:15:01.311482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:15:01.311482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 14:15:01.311482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:01.314391 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 376/755] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:15:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:01.322183 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a1fe2-8d8b-4bf6-ad4d-17c26bf9fb76 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:01.383107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50e083-3a00-4236-b22e-5c9eb4898e61 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:01.395692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:01.424692 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d5de6-299d-4935-a59d-3525a93c052c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:01.425309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d5de6-299d-4935-a59d-3525a93c052c None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:01.425605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d5de6-299d-4935-a59d-3525a93c052c None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:01.425867 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d5de6-299d-4935-a59d-3525a93c052c None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:01.426519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d5de6-299d-4935-a59d-3525a93c052c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:01.426891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d5de6-299d-4935-a59d-3525a93c052c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:01.428294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d5de6-299d-4935-a59d-3525a93c052c None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:01.443918 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 380/756] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:01.448954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d5de6-299d-4935-a59d-3525a93c052c None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:01.456900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a1fe2-8d8b-4bf6-ad4d-17c26bf9fb76 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:01.515423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:01.520694 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7229e1-8d87-4345-9e7f-f487db5d3efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:01.521323 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7229e1-8d87-4345-9e7f-f487db5d3efe None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:01.521609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7229e1-8d87-4345-9e7f-f487db5d3efe None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:01.521911 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7229e1-8d87-4345-9e7f-f487db5d3efe None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:01.522652 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7229e1-8d87-4345-9e7f-f487db5d3efe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:01.522987 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7229e1-8d87-4345-9e7f-f487db5d3efe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:01.524953 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7229e1-8d87-4345-9e7f-f487db5d3efe None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:01.565360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 377/757] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Sep 5 14:15:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:01.576525 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7229e1-8d87-4345-9e7f-f487db5d3efe None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:01.580386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d5de6-299d-4935-a59d-3525a93c052c None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:01.644651 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c61a9-c3e2-41e2-99a6-c8de87ad5978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:01.645223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c61a9-c3e2-41e2-99a6-c8de87ad5978 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:01.645510 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c61a9-c3e2-41e2-99a6-c8de87ad5978 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:01.645791 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c61a9-c3e2-41e2-99a6-c8de87ad5978 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:01.661326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c61a9-c3e2-41e2-99a6-c8de87ad5978 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:01.662832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c61a9-c3e2-41e2-99a6-c8de87ad5978 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:01.681513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-f31c61a9-c3e2-41e2-99a6-c8de87ad5978 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:01.681513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:01.681513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:01.681513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:01.681513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:01.681513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:01.681513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:01.681513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:01.681513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:01.681513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:01.681513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:01.681513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:01.681513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:15:01.681513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:15:01.681513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:15:01.682489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:15:01.682489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:01.682489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:01.682489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:01.682489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:01.682489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:01.682489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:01.682489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:01.682489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:01.682489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:01.682489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:01.682489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:01.682489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:01.682489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:01.682489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:01.682489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:01.682489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:01.683439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:15:01.683439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:15:01.683439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:15:01.683439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:15:01.683439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:15:01.683439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:15:01.683439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:01.683439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:01.691118 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 381/758] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:01.693000 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c61a9-c3e2-41e2-99a6-c8de87ad5978 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:01.750046 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7229e1-8d87-4345-9e7f-f487db5d3efe None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:01.802963 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f9f23-3340-41ce-9ef7-0ff1143e3005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:01.803604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f9f23-3340-41ce-9ef7-0ff1143e3005 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:01.803886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f9f23-3340-41ce-9ef7-0ff1143e3005 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:01.804161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f9f23-3340-41ce-9ef7-0ff1143e3005 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:01.805256 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c61a9-c3e2-41e2-99a6-c8de87ad5978 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:01.807977 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:01.811119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:01.846963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d8328-f92f-4d08-8b74-a38c76b730c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:01.847813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d8328-f92f-4d08-8b74-a38c76b730c4 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:01.848365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d8328-f92f-4d08-8b74-a38c76b730c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:01.848808 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d8328-f92f-4d08-8b74-a38c76b730c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:01.849664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d8328-f92f-4d08-8b74-a38c76b730c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:01.850163 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d8328-f92f-4d08-8b74-a38c76b730c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:01.851781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d8328-f92f-4d08-8b74-a38c76b730c4 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:01.867387 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 382/759] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:15:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:01.869192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847a473-6e4f-4f6e-a8f7-7863a3cf4309 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:01.869763 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847a473-6e4f-4f6e-a8f7-7863a3cf4309 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:01.870254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847a473-6e4f-4f6e-a8f7-7863a3cf4309 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:01.872016 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 383/760] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:01.873627 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847a473-6e4f-4f6e-a8f7-7863a3cf4309 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:01.896528 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f9f23-3340-41ce-9ef7-0ff1143e3005 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:01.898301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f9f23-3340-41ce-9ef7-0ff1143e3005 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:01.998943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847a473-6e4f-4f6e-a8f7-7863a3cf4309 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:02.032011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 378/761] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:15:01 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 14:15:02.033602 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb623b-ce32-4ef8-acea-9dadf3a67a41 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:02.034239 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb623b-ce32-4ef8-acea-9dadf3a67a41 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:02.034740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb623b-ce32-4ef8-acea-9dadf3a67a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:02.045235 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95622135-d4e8-4f66-a3eb-b92cb04a11ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:02.046446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95622135-d4e8-4f66-a3eb-b92cb04a11ff None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:02.046955 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95622135-d4e8-4f66-a3eb-b92cb04a11ff None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:02.047422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95622135-d4e8-4f66-a3eb-b92cb04a11ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:02.048336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95622135-d4e8-4f66-a3eb-b92cb04a11ff None admin] RBAC: Adding query filter params (role.id=0a83e40e2d64470a8489bf548e34dabd, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=86300433c724483994f59709a3f015db) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:02.048864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95622135-d4e8-4f66-a3eb-b92cb04a11ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:02.050466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95622135-d4e8-4f66-a3eb-b92cb04a11ff None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:02.057448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 384/762] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:15:01 2022] GET /identity/v3/role_assignments?user.id=86300433c724483994f59709a3f015db&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=0a83e40e2d64470a8489bf548e34dabd => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:02.057678 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-bfbb623b-ce32-4ef8-acea-9dadf3a67a41 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:02.058665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95622135-d4e8-4f66-a3eb-b92cb04a11ff None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:02.059042 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-bfbb623b-ce32-4ef8-acea-9dadf3a67a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:02.075190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:02.105339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:02.142925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 379/763] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:02.190425 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95622135-d4e8-4f66-a3eb-b92cb04a11ff None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:02.216077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b4b43-2847-46c7-9116-d087c1d11216 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:02.216401 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b4b43-2847-46c7-9116-d087c1d11216 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:02.216636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b4b43-2847-46c7-9116-d087c1d11216 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:02.218365 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 380/764] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:15:02.223643 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a412d-0938-4d33-9287-32773070d1a6 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:02.223943 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a412d-0938-4d33-9287-32773070d1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:02.224194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a412d-0938-4d33-9287-32773070d1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:02.237164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb355c-2b18-4753-a381-548137d8536c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:02.238166 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb355c-2b18-4753-a381-548137d8536c None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:02.239410 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb355c-2b18-4753-a381-548137d8536c None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:02.239972 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb355c-2b18-4753-a381-548137d8536c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:02.252237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-0d1a412d-0938-4d33-9287-32773070d1a6 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:02.253697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-0d1a412d-0938-4d33-9287-32773070d1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:02.263905 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb355c-2b18-4753-a381-548137d8536c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:02.265776 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb355c-2b18-4753-a381-548137d8536c None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:02.281847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-e2bb355c-2b18-4753-a381-548137d8536c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:02.281847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:02.281847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:02.281847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:02.281847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:02.281847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:02.281847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:02.281847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:02.281847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:02.281847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:02.281847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:02.281847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:02.281847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:02.281847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:02.281847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:02.281847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:02.282833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:02.282833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:02.282833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:02.282833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:02.282833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:02.282833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:02.282833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:02.282833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:02.282833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:02.282833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:02.282833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:02.282833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:02.282833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:02.282833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:02.282833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:02.282833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:02.282833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:02.283755 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:02.283755 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:02.283755 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:02.285832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 385/765] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:02.292217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb355c-2b18-4753-a381-548137d8536c None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:02.383187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:02.407033 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb355c-2b18-4753-a381-548137d8536c None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:02.429107 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 381/766] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:02.449990 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89004c6a-0d8e-46ee-a43a-3b4dd619f406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:02.450902 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89004c6a-0d8e-46ee-a43a-3b4dd619f406 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:02.451361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89004c6a-0d8e-46ee-a43a-3b4dd619f406 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:02.451836 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89004c6a-0d8e-46ee-a43a-3b4dd619f406 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:02.452663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89004c6a-0d8e-46ee-a43a-3b4dd619f406 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:02.453139 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89004c6a-0d8e-46ee-a43a-3b4dd619f406 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:02.454698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89004c6a-0d8e-46ee-a43a-3b4dd619f406 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:02.471960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 386/767] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:02.473588 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a412d-0938-4d33-9287-32773070d1a6 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:02.499387 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89004c6a-0d8e-46ee-a43a-3b4dd619f406 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:02.575358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:02.606557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89004c6a-0d8e-46ee-a43a-3b4dd619f406 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:02.614201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a412d-0938-4d33-9287-32773070d1a6 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:02.658584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683db9c-a3c0-4aea-90c4-81f567304ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:02.659251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683db9c-a3c0-4aea-90c4-81f567304ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:02.659529 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683db9c-a3c0-4aea-90c4-81f567304ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:02.659798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683db9c-a3c0-4aea-90c4-81f567304ca8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:02.660374 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683db9c-a3c0-4aea-90c4-81f567304ca8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:02.661739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683db9c-a3c0-4aea-90c4-81f567304ca8 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:02.675759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-b683db9c-a3c0-4aea-90c4-81f567304ca8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 14:15:02.675759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:02.675759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:02.675759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:02.675759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:02.675759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:02.675759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:02.675759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:02.675759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 14:15:02.675759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:02.675759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:02.675759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:02.675759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:02.675759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:02.675759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:02.675759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:02.675759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:02.676859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:02.676859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:02.676859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:02.676859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:02.676859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:02.676859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:02.676859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:02.676859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:02.676859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:02.676859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:02.676859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:02.676859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:02.676859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:02.676859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:02.676859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:02.676859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:02.677716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:02.677716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:02.677716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:02.677716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:02.677716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:02.677716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:02.677716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:02.677716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:02.677716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:02.677716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:02.677716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:02.677716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:02.677716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 14:15:02.677716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:02.677716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 387/768] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:02 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:02.678526 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450106e6-5231-456d-bcef-3e001952854d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:02.678526 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450106e6-5231-456d-bcef-3e001952854d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:02.678678 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450106e6-5231-456d-bcef-3e001952854d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:02.679316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450106e6-5231-456d-bcef-3e001952854d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:02.682917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683db9c-a3c0-4aea-90c4-81f567304ca8 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:02.687103 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:02.693295 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450106e6-5231-456d-bcef-3e001952854d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:02.694858 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450106e6-5231-456d-bcef-3e001952854d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:02.717609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-450106e6-5231-456d-bcef-3e001952854d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:02.717609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:02.717609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:02.717609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:02.717609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:02.717609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:02.717609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:02.717609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:02.717609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:02.717609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:02.717609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:02.717609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:02.717609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:15:02.717609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:15:02.717609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:15:02.718560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:15:02.718560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:02.718560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:02.718560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:02.718560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:02.718560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:02.718560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:02.718560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:02.718560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:02.718560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:02.718560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:02.718560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:02.718560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:02.718560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:02.718560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:02.718560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:02.718560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:02.719501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:15:02.719501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:15:02.719501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:15:02.719501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:15:02.719501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:15:02.719501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:15:02.719501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:02.719501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:02.723467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 382/769] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:02.724706 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450106e6-5231-456d-bcef-3e001952854d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:02.785697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683db9c-a3c0-4aea-90c4-81f567304ca8 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:02.822811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc11db-6aca-414a-9119-f2592df540df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:02.822811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc11db-6aca-414a-9119-f2592df540df None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:02.822811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc11db-6aca-414a-9119-f2592df540df None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:02.822811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc11db-6aca-414a-9119-f2592df540df None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:02.823228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc11db-6aca-414a-9119-f2592df540df None admin] RBAC: Adding query filter params (name=network) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:02.823228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc11db-6aca-414a-9119-f2592df540df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:02.823379 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc11db-6aca-414a-9119-f2592df540df None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:02.828482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 388/770] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 5 14:15:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:02.833932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc11db-6aca-414a-9119-f2592df540df None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:02.879562 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450106e6-5231-456d-bcef-3e001952854d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:02.902719 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:02.917787 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030e082-75b5-4d6e-bd23-0cc63e3bd6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:02.918125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030e082-75b5-4d6e-bd23-0cc63e3bd6b6 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:02.918409 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030e082-75b5-4d6e-bd23-0cc63e3bd6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:02.918680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030e082-75b5-4d6e-bd23-0cc63e3bd6b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:02.919405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030e082-75b5-4d6e-bd23-0cc63e3bd6b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:02.919710 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030e082-75b5-4d6e-bd23-0cc63e3bd6b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:02.921152 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030e082-75b5-4d6e-bd23-0cc63e3bd6b6 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:02.923752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:02.939567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 383/771] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:15:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:02.940494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030e082-75b5-4d6e-bd23-0cc63e3bd6b6 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:02.941501 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc11db-6aca-414a-9119-f2592df540df None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:02.978339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6d1ad-2cf4-411c-add6-4c327d7728af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:02.979395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6d1ad-2cf4-411c-add6-4c327d7728af None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:02.979850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6d1ad-2cf4-411c-add6-4c327d7728af None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:02.980450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6d1ad-2cf4-411c-add6-4c327d7728af None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:02.981245 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6d1ad-2cf4-411c-add6-4c327d7728af None admin] RBAC: Adding query filter params (type=network) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:02.981728 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6d1ad-2cf4-411c-add6-4c327d7728af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:02.983402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6d1ad-2cf4-411c-add6-4c327d7728af None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:02.989156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 389/772] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 5 14:15:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:02.995670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6d1ad-2cf4-411c-add6-4c327d7728af None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:03.048838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030e082-75b5-4d6e-bd23-0cc63e3bd6b6 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:03.086537 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd271d-3945-4218-879e-0003144e4c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:03.087168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd271d-3945-4218-879e-0003144e4c6b None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:03.087449 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd271d-3945-4218-879e-0003144e4c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:03.087720 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd271d-3945-4218-879e-0003144e4c6b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:03.088316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd271d-3945-4218-879e-0003144e4c6b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:03.089923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd271d-3945-4218-879e-0003144e4c6b None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:03.107825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-2fcd271d-3945-4218-879e-0003144e4c6b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 14:15:03.107825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:03.107825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:03.107825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:03.107825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:03.107825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:03.107825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:03.107825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:03.107825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 14:15:03.107825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:03.107825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:03.107825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:03.107825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:03.107825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:03.107825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:03.107825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:03.107825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:03.108920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:03.108920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:03.108920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:03.108920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:03.108920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:03.108920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:03.108920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:03.108920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:03.108920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:03.108920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:03.108920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:03.108920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:03.108920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:03.108920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:03.108920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:03.108920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:03.109775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:03.109775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:03.109775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:03.109775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:03.109775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:03.109775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:03.109775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:03.109775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:03.109775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:03.109775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:03.109775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:03.109775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:03.109775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 14:15:03.109775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:03.113945 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 384/773] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Sep 5 14:15:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:03.114780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd271d-3945-4218-879e-0003144e4c6b None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:03.119979 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6d1ad-2cf4-411c-add6-4c327d7728af None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:03.152324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:03.158705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e281c-5647-4dde-9639-62b19d89927a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:03.159715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e281c-5647-4dde-9639-62b19d89927a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:03.160161 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e281c-5647-4dde-9639-62b19d89927a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:03.160593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e281c-5647-4dde-9639-62b19d89927a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:03.166401 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e281c-5647-4dde-9639-62b19d89927a None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=0a85809d14cb449d848f22a3f39bf641) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:03.166401 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e281c-5647-4dde-9639-62b19d89927a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:03.166401 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e281c-5647-4dde-9639-62b19d89927a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:03.169841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 390/774] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:15:02 2022] GET /identity/v3/role_assignments?user.id=0a85809d14cb449d848f22a3f39bf641&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:03.170652 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e281c-5647-4dde-9639-62b19d89927a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:03.196917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:03.291237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e281c-5647-4dde-9639-62b19d89927a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:03.293114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd271d-3945-4218-879e-0003144e4c6b None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:03.331171 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d70c-8436-4d07-968b-99d240d6f68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:03.331810 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d70c-8436-4d07-968b-99d240d6f68e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:03.332119 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d70c-8436-4d07-968b-99d240d6f68e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:03.332350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d70c-8436-4d07-968b-99d240d6f68e None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:03.333009 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201d70c-8436-4d07-968b-99d240d6f68e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:03.333312 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201d70c-8436-4d07-968b-99d240d6f68e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:03.334810 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201d70c-8436-4d07-968b-99d240d6f68e None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:03.341060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 391/775] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Sep 5 14:15:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:03.348296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d70c-8436-4d07-968b-99d240d6f68e None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:03.388200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d363840-04a3-4bbd-bbe1-c375aeb270ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:03.389076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d363840-04a3-4bbd-bbe1-c375aeb270ab None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:03.389076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d363840-04a3-4bbd-bbe1-c375aeb270ab None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:03.389392 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d363840-04a3-4bbd-bbe1-c375aeb270ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:03.390185 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d363840-04a3-4bbd-bbe1-c375aeb270ab None admin] RBAC: Adding query filter params (interface=public, service_id=edc74bbfcc0e43cdbf60acf230d9872e, region_id=RegionOne) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:03.390599 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d363840-04a3-4bbd-bbe1-c375aeb270ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:03.392027 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d363840-04a3-4bbd-bbe1-c375aeb270ab None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:03.400802 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 385/776] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Sep 5 14:15:03 2022] GET /identity/v3/endpoints?service_id=edc74bbfcc0e43cdbf60acf230d9872e&interface=public®ion_id=RegionOne => generated 194 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:03.410373 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d363840-04a3-4bbd-bbe1-c375aeb270ab None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:03.454522 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:03.546471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d70c-8436-4d07-968b-99d240d6f68e None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:03.561418 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d363840-04a3-4bbd-bbe1-c375aeb270ab None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:03.603270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:03.618562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd09c98-52b0-4f05-bd5e-a0343786cdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:03.620845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd09c98-52b0-4f05-bd5e-a0343786cdf2 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:03.620845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd09c98-52b0-4f05-bd5e-a0343786cdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:03.620845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd09c98-52b0-4f05-bd5e-a0343786cdf2 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:03.620845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd09c98-52b0-4f05-bd5e-a0343786cdf2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:03.620845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd09c98-52b0-4f05-bd5e-a0343786cdf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:03.622247 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd09c98-52b0-4f05-bd5e-a0343786cdf2 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:03.625819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e875ba-3599-4b47-a5a5-f9afd299c1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:03.626673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e875ba-3599-4b47-a5a5-f9afd299c1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:03.627259 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e875ba-3599-4b47-a5a5-f9afd299c1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:03.627832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e875ba-3599-4b47-a5a5-f9afd299c1d3 None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:03.628023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 392/777] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Sep 5 14:15:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:03.628920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e875ba-3599-4b47-a5a5-f9afd299c1d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:03.629175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b22c2-77f2-45dd-8e7b-629fd269b08b None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:03.629461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b22c2-77f2-45dd-8e7b-629fd269b08b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:03.629737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b22c2-77f2-45dd-8e7b-629fd269b08b None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:03.631385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e875ba-3599-4b47-a5a5-f9afd299c1d3 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:03.631515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 393/778] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:03.636694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b22c2-77f2-45dd-8e7b-629fd269b08b None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:03.639559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 386/779] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 5 14:15:03 2022] GET /identity/v3/services => generated 772 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:03.640897 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3924dd3-f901-411b-8c16-249120dba965 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:03.641188 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3924dd3-f901-411b-8c16-249120dba965 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:03.641465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3924dd3-f901-411b-8c16-249120dba965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:03.662746 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-c3924dd3-f901-411b-8c16-249120dba965 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:03.664179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c3924dd3-f901-411b-8c16-249120dba965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:03.776627 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 387/780] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:03.808500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b22c2-77f2-45dd-8e7b-629fd269b08b None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:03.834838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:03.848550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6280e89-45cc-459b-85e5-b85e977cf846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:03.849624 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6280e89-45cc-459b-85e5-b85e977cf846 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:03.849624 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6280e89-45cc-459b-85e5-b85e977cf846 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:03.849807 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6280e89-45cc-459b-85e5-b85e977cf846 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:03.850397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6280e89-45cc-459b-85e5-b85e977cf846 None admin] RBAC: Adding query filter params (interface=public, service_id=20963990208b49aabe54acb13d3ab75c, region_id=RegionOne) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:03.850706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6280e89-45cc-459b-85e5-b85e977cf846 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:03.852104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6280e89-45cc-459b-85e5-b85e977cf846 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:03.866593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 394/781] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Sep 5 14:15:03 2022] GET /identity/v3/endpoints?service_id=20963990208b49aabe54acb13d3ab75c&interface=public®ion_id=RegionOne => generated 194 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:03.870153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3924dd3-f901-411b-8c16-249120dba965 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:03.973988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3924dd3-f901-411b-8c16-249120dba965 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:03.993003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:04.010661 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77173b47-aec0-43d6-9d0e-2ea6bdcc5310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:04.011384 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77173b47-aec0-43d6-9d0e-2ea6bdcc5310 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:04.011730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77173b47-aec0-43d6-9d0e-2ea6bdcc5310 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:04.012079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77173b47-aec0-43d6-9d0e-2ea6bdcc5310 None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:04.012752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77173b47-aec0-43d6-9d0e-2ea6bdcc5310 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:04.014197 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77173b47-aec0-43d6-9d0e-2ea6bdcc5310 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:04.019566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 388/782] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 5 14:15:03 2022] GET /identity/v3/services => generated 772 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:04.117777 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6280e89-45cc-459b-85e5-b85e977cf846 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:04.242377 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6280e89-45cc-459b-85e5-b85e977cf846 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:04.278446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd0636-c7fa-4636-8127-18427d57e66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:04.279127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd0636-c7fa-4636-8127-18427d57e66e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:04.279408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd0636-c7fa-4636-8127-18427d57e66e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:04.279683 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd0636-c7fa-4636-8127-18427d57e66e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:04.289446 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f86cff-2e33-4e3c-a5de-2f96a4110f48 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:04.289733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f86cff-2e33-4e3c-a5de-2f96a4110f48 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:04.290541 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f86cff-2e33-4e3c-a5de-2f96a4110f48 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:04.291833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd0636-c7fa-4636-8127-18427d57e66e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:15:04.296573 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 389/783] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:04.296708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-e4dd0636-c7fa-4636-8127-18427d57e66e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:15:04.296708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:04.296708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:04.296708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:04.296708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:04.296708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:04.296708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:04.296708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:04.296708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:04.296708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:04.296708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:04.296708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:04.296708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:15:04.296708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:15:04.296708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:15:04.296708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:15:04.297585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:15:04.297585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:15:04.297585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:15:04.297585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:15:04.297585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:04.297585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:04.297585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:04.297585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:04.297585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:04.297585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:04.297585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:04.297585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:04.297585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:04.297585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:04.297585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:04.297585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:04.298445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:04.298445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:04.298445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:04.298445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:04.298445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:15:04.298445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:15:04.298445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:15:04.298445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:15:04.298445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:15:04.298445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:15:04.298445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:15:04.298445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:04.298445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 395/784] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:15:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:04.299414 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54634e-0750-4578-bc63-b7526edbc3f0 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:04.299414 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54634e-0750-4578-bc63-b7526edbc3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:04.299414 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54634e-0750-4578-bc63-b7526edbc3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:04.303790 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd0636-c7fa-4636-8127-18427d57e66e None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:04.325539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-bf54634e-0750-4578-bc63-b7526edbc3f0 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:04.327817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-bf54634e-0750-4578-bc63-b7526edbc3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:04.395234 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:04.460589 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 390/785] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:04.474023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd0636-c7fa-4636-8127-18427d57e66e None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:04.522726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9708e83-6da3-4cf3-882f-244bf57b7b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:04.523645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9708e83-6da3-4cf3-882f-244bf57b7b27 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:04.524150 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9708e83-6da3-4cf3-882f-244bf57b7b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:04.525114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9708e83-6da3-4cf3-882f-244bf57b7b27 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:04.525929 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9708e83-6da3-4cf3-882f-244bf57b7b27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:04.526382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9708e83-6da3-4cf3-882f-244bf57b7b27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:04.527943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9708e83-6da3-4cf3-882f-244bf57b7b27 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:04.543242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 396/786] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:15:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:04.548434 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54634e-0750-4578-bc63-b7526edbc3f0 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:04.709612 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54634e-0750-4578-bc63-b7526edbc3f0 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:04.727920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:04.752434 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74c702-04e7-4f75-a963-fedc60933af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:04.753076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74c702-04e7-4f75-a963-fedc60933af3 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:04.753360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74c702-04e7-4f75-a963-fedc60933af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:04.753633 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74c702-04e7-4f75-a963-fedc60933af3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:04.764024 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74c702-04e7-4f75-a963-fedc60933af3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:04.765828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74c702-04e7-4f75-a963-fedc60933af3 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:04.785036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-cd74c702-04e7-4f75-a963-fedc60933af3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:04.785036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:04.785036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:04.785036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:04.785036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:04.785036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:04.785036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:04.785036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:04.785036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:04.785036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:04.785036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:04.785036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:04.785036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:04.785036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:04.785036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:04.785036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:04.786010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:04.786010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:04.786010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:04.786010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:04.786010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:04.786010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:04.786010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:04.786010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:04.786010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:04.786010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:04.786010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:04.786010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:04.786010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:04.786010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:04.786010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:04.786010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:04.786010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:04.786995 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:04.786995 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:04.786995 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:04.787181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 391/787] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:04.792310 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9708e83-6da3-4cf3-882f-244bf57b7b27 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:04.793822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74c702-04e7-4f75-a963-fedc60933af3 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:04.940277 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74c702-04e7-4f75-a963-fedc60933af3 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:04.980094 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0e504-eeac-42d0-b7d3-80c1ca248e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:04.980732 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0e504-eeac-42d0-b7d3-80c1ca248e44 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:04.981012 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0e504-eeac-42d0-b7d3-80c1ca248e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:04.981283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0e504-eeac-42d0-b7d3-80c1ca248e44 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:04.985896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9708e83-6da3-4cf3-882f-244bf57b7b27 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:05.001298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0e504-eeac-42d0-b7d3-80c1ca248e44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:15:05.006564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-6eb0e504-eeac-42d0-b7d3-80c1ca248e44 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:15:05.006564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:05.006564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:05.006564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:05.006564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:05.006564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:05.006564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:05.006564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:05.006564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:05.006564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:05.006564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:05.006564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:05.006564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:15:05.006564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:15:05.006564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:15:05.006564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:15:05.007560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:15:05.007560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:15:05.007560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:15:05.007560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:15:05.007560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:05.007560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:05.007560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:05.007560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:05.007560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:05.007560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:05.007560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:05.007560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:05.007560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:05.007560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:05.007560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:05.007560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:05.008424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:05.008424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:05.008424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:05.008424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:05.008424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:15:05.008424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:15:05.008424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:15:05.008424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:15:05.008424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:15:05.008424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:15:05.008424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:15:05.008424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:05.009622 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:05.014210 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 392/788] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:15:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:05.020716 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0e504-eeac-42d0-b7d3-80c1ca248e44 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:05.051146 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:05.055525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02a9da-829b-490a-99ca-4fc9b4f32a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:05.056160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02a9da-829b-490a-99ca-4fc9b4f32a71 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:05.056444 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02a9da-829b-490a-99ca-4fc9b4f32a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:05.056715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02a9da-829b-490a-99ca-4fc9b4f32a71 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:05.057365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02a9da-829b-490a-99ca-4fc9b4f32a71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:05.057668 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02a9da-829b-490a-99ca-4fc9b4f32a71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:05.059089 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02a9da-829b-490a-99ca-4fc9b4f32a71 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:05.096336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 397/789] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:05.101959 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02a9da-829b-490a-99ca-4fc9b4f32a71 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:05.131128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0e504-eeac-42d0-b7d3-80c1ca248e44 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:05.167196 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f9bca-e05c-4690-bd04-f845b42091eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:05.168164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f9bca-e05c-4690-bd04-f845b42091eb None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:05.168741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f9bca-e05c-4690-bd04-f845b42091eb None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:05.169209 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f9bca-e05c-4690-bd04-f845b42091eb None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:05.170074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f9bca-e05c-4690-bd04-f845b42091eb None admin] RBAC: Adding query filter params (name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:05.170569 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f9bca-e05c-4690-bd04-f845b42091eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:05.172294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f9bca-e05c-4690-bd04-f845b42091eb None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:05.182312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 393/790] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:15:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:05.188248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f9bca-e05c-4690-bd04-f845b42091eb None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:05.281574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:05.289843 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02a9da-829b-490a-99ca-4fc9b4f32a71 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:05.297967 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f9bca-e05c-4690-bd04-f845b42091eb None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:05.334391 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2b88e-cb05-4f6b-805b-5d1af5347da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:05.335400 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2b88e-cb05-4f6b-805b-5d1af5347da6 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:05.335956 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2b88e-cb05-4f6b-805b-5d1af5347da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:05.336448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2b88e-cb05-4f6b-805b-5d1af5347da6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:05.347483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc9aad-dc6a-421c-8e9b-baa516befc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:05.348451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc9aad-dc6a-421c-8e9b-baa516befc13 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:05.348926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc9aad-dc6a-421c-8e9b-baa516befc13 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:05.349390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc9aad-dc6a-421c-8e9b-baa516befc13 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:05.356021 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2b88e-cb05-4f6b-805b-5d1af5347da6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:05.361098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2b88e-cb05-4f6b-805b-5d1af5347da6 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:05.363226 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc9aad-dc6a-421c-8e9b-baa516befc13 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:05.365874 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc9aad-dc6a-421c-8e9b-baa516befc13 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:05.391890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-5afc9aad-dc6a-421c-8e9b-baa516befc13 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:15:05.391890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:05.391890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:05.391890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:05.391890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:05.391890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:05.391890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:05.391890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:05.391890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:05.391890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:05.391890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:05.391890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:05.391890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:15:05.391890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:15:05.391890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:15:05.391890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:05.392893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:05.393844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:05.393844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:05.393844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:15:05.393844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:15:05.393844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:15:05.393844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:15:05.393844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:15:05.393844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:15:05.393844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:15:05.393844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:05.395184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-bfe2b88e-cb05-4f6b-805b-5d1af5347da6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:05.395184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:05.395184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:05.395184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:05.395184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:05.395184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:05.395184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:05.395184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:05.395184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:05.395184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:05.395184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:05.395184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:05.395184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:05.395184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:05.395184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:05.395184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:05.396067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:05.396067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:05.396067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:05.396067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:05.396067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:05.396067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:05.396067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:05.396067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:05.396067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:05.396067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:05.396067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:05.396067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:05.396067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:05.396067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:05.396067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:05.396067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:05.396067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:05.396983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:05.396983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:05.396983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:05.398690 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 398/791] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 5 14:15:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:05.400864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 394/792] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:05.405230 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc9aad-dc6a-421c-8e9b-baa516befc13 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:05.408181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2b88e-cb05-4f6b-805b-5d1af5347da6 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:05.443350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:05.549418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc9aad-dc6a-421c-8e9b-baa516befc13 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:05.589469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2b88e-cb05-4f6b-805b-5d1af5347da6 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:05.593021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b27332-5a8b-4ecc-829f-afb78f670c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:05.593939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b27332-5a8b-4ecc-829f-afb78f670c51 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:05.594414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b27332-5a8b-4ecc-829f-afb78f670c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:05.594948 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b27332-5a8b-4ecc-829f-afb78f670c51 None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:05.595875 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b27332-5a8b-4ecc-829f-afb78f670c51 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:05.596393 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b27332-5a8b-4ecc-829f-afb78f670c51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:05.598414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b27332-5a8b-4ecc-829f-afb78f670c51 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:05.621799 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 399/793] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 5 14:15:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:05.624306 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc5f35-9091-451e-b362-490f7818acbe None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:05.624842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc5f35-9091-451e-b362-490f7818acbe None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:05.625258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc5f35-9091-451e-b362-490f7818acbe None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:05.627150 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 400/794] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:05.628007 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc5f35-9091-451e-b362-490f7818acbe None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:05.633041 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33e66c-02e4-4a5a-97e6-a0192fb4fccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:05.633822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33e66c-02e4-4a5a-97e6-a0192fb4fccb None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:05.634210 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33e66c-02e4-4a5a-97e6-a0192fb4fccb None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:05.634588 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33e66c-02e4-4a5a-97e6-a0192fb4fccb None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:05.635376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33e66c-02e4-4a5a-97e6-a0192fb4fccb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:05.635787 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33e66c-02e4-4a5a-97e6-a0192fb4fccb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:05.637267 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33e66c-02e4-4a5a-97e6-a0192fb4fccb None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:05.646690 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:05.655792 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 395/795] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:05.658152 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9ae29-9210-4639-83a2-bacbe23f7cd9 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:05.658152 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9ae29-9210-4639-83a2-bacbe23f7cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:05.658152 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9ae29-9210-4639-83a2-bacbe23f7cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:05.678516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-f1d9ae29-9210-4639-83a2-bacbe23f7cd9 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:05.680166 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f1d9ae29-9210-4639-83a2-bacbe23f7cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:05.726408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:05.746537 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc5f35-9091-451e-b362-490f7818acbe None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:05.763077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 396/796] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:05.764278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9ae29-9210-4639-83a2-bacbe23f7cd9 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:05.786974 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793d6e0-b832-4fcf-9084-ba9b6b9d1d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:05.788141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793d6e0-b832-4fcf-9084-ba9b6b9d1d9a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:05.788704 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793d6e0-b832-4fcf-9084-ba9b6b9d1d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:05.789111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793d6e0-b832-4fcf-9084-ba9b6b9d1d9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:05.871664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793d6e0-b832-4fcf-9084-ba9b6b9d1d9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:05.871664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793d6e0-b832-4fcf-9084-ba9b6b9d1d9a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:05.955267 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9ae29-9210-4639-83a2-bacbe23f7cd9 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:06.003214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:06.012095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:06.024734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 401/797] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:15:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:15:06.027226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f5d9f-23d0-49fa-8d68-a19ce0c488ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:06.027863 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f5d9f-23d0-49fa-8d68-a19ce0c488ff None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:06.028181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f5d9f-23d0-49fa-8d68-a19ce0c488ff None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:06.028457 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f5d9f-23d0-49fa-8d68-a19ce0c488ff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:06.031239 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793d6e0-b832-4fcf-9084-ba9b6b9d1d9a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:06.047976 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f5d9f-23d0-49fa-8d68-a19ce0c488ff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:06.049571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f5d9f-23d0-49fa-8d68-a19ce0c488ff None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:06.072443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-994f5d9f-23d0-49fa-8d68-a19ce0c488ff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 14:15:06.072443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:06.072443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:06.072443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:06.072443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:06.072443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:06.072443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:06.072443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:06.072443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:06.072443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:06.072443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:06.072443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:06.072443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:15:06.072443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:15:06.072443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:15:06.072443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:06.073479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:06.074465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:06.074465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:06.074465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:15:06.074465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:15:06.074465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:15:06.074465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:15:06.074465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:15:06.074465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:15:06.074465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 14:15:06.074465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:06.087543 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 397/798] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 5 14:15:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:06.095881 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f5d9f-23d0-49fa-8d68-a19ce0c488ff None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:06.250578 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f5d9f-23d0-49fa-8d68-a19ce0c488ff None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:06.257059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793d6e0-b832-4fcf-9084-ba9b6b9d1d9a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:06.285869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b1556-5920-4b86-86f4-4cfecf40effc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:06.286475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b1556-5920-4b86-86f4-4cfecf40effc None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:06.286750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b1556-5920-4b86-86f4-4cfecf40effc None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:06.287052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b1556-5920-4b86-86f4-4cfecf40effc None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:06.287771 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b1556-5920-4b86-86f4-4cfecf40effc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:06.288084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b1556-5920-4b86-86f4-4cfecf40effc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:06.289487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b1556-5920-4b86-86f4-4cfecf40effc None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:06.292612 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a6dff-8d96-4624-bbf7-053dfeb467f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:06.293089 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a6dff-8d96-4624-bbf7-053dfeb467f4 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:06.293170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a6dff-8d96-4624-bbf7-053dfeb467f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:06.293386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a6dff-8d96-4624-bbf7-053dfeb467f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:06.304449 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0a6dff-8d96-4624-bbf7-053dfeb467f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:06.305583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0a6dff-8d96-4624-bbf7-053dfeb467f4 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:06.312158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 398/799] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Sep 5 14:15:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:06.312158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b1556-5920-4b86-86f4-4cfecf40effc None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:06.325714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-4b0a6dff-8d96-4624-bbf7-053dfeb467f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:06.325714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:06.325714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:06.325714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:06.325714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:06.325714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:06.325714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:06.325714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:06.325714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:06.325714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:06.325714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:06.325714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:06.325714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:06.325714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:06.325714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:06.325714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:06.326819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:06.326819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:06.326819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:06.326819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:06.326819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:06.326819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:06.326819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:06.326819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:06.326819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:06.326819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:06.326819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:06.326819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:06.326819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:06.326819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:06.326819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:06.326819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:06.326819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:06.327739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:06.327739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:06.327739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:06.327739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 402/800] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:06.327979 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec727477-261c-48b2-929b-d73ba59cb212 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:06.328477 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec727477-261c-48b2-929b-d73ba59cb212 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:06.328575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec727477-261c-48b2-929b-d73ba59cb212 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:06.333848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 403/801] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:06.333848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec727477-261c-48b2-929b-d73ba59cb212 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:06.365209 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:06.378334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:06.418261 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b1556-5920-4b86-86f4-4cfecf40effc None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:06.438147 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec727477-261c-48b2-929b-d73ba59cb212 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:06.451595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e51ed2-8990-4409-a199-7fee1851d27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:06.452211 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e51ed2-8990-4409-a199-7fee1851d27e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:06.452491 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e51ed2-8990-4409-a199-7fee1851d27e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:06.452759 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e51ed2-8990-4409-a199-7fee1851d27e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:06.462577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e51ed2-8990-4409-a199-7fee1851d27e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:15:06.467707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-e6e51ed2-8990-4409-a199-7fee1851d27e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:15:06.467707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:06.467707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:06.467707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:06.467707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:06.467707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:06.467707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:06.467707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:06.467707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:06.467707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:06.467707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:06.467707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:06.467707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:15:06.467707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:15:06.467707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:15:06.467707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:15:06.468640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:15:06.468640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:15:06.468640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:15:06.468640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:15:06.468640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:06.468640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:06.468640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:06.468640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:06.468640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:06.468640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:06.468640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:06.468640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:06.468640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:06.468640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:06.468640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:06.468640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:06.469493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:06.469493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:06.469493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:06.469493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:06.469493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:15:06.469493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:15:06.469493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:15:06.469493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:15:06.469493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:15:06.469493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:15:06.469493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:15:06.469493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:06.469493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 399/802] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:15:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:06.470190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e51ed2-8990-4409-a199-7fee1851d27e None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:06.473238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf8aad-374b-4427-af25-eb68964c791b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:06.473997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf8aad-374b-4427-af25-eb68964c791b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:06.474272 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf8aad-374b-4427-af25-eb68964c791b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:06.474546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf8aad-374b-4427-af25-eb68964c791b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:06.485236 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf8aad-374b-4427-af25-eb68964c791b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:06.486693 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf8aad-374b-4427-af25-eb68964c791b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:06.501182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-3acf8aad-374b-4427-af25-eb68964c791b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:06.501182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:06.501182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:06.501182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:06.501182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:06.501182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:06.501182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:06.501182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:06.501182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:06.501182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:06.501182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:06.501182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:06.501182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:06.501182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:06.501182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:06.501182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:06.502293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:06.502293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:06.502293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:06.502293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:06.502293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:06.502293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:06.502293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:06.502293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:06.502293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:06.502293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:06.502293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:06.502293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:06.502293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:06.502293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:06.502293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:06.502293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:06.502293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:06.503733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:06.503733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:06.503733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:06.503733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 404/803] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:06.503733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f395ae4-a4af-4489-a292-b18cdb95d006 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:06.503733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f395ae4-a4af-4489-a292-b18cdb95d006 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:06.504160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f395ae4-a4af-4489-a292-b18cdb95d006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:06.521393 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-7f395ae4-a4af-4489-a292-b18cdb95d006 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:06.522581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7f395ae4-a4af-4489-a292-b18cdb95d006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:06.575629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e51ed2-8990-4409-a199-7fee1851d27e None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:06.582384 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:06.587428 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 405/804] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:06.588642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49443036-57d9-46e4-bf12-ff3be8d0020a None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:06.588927 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49443036-57d9-46e4-bf12-ff3be8d0020a None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:06.589205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49443036-57d9-46e4-bf12-ff3be8d0020a None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:06.590822 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 406/805] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:06.591340 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49443036-57d9-46e4-bf12-ff3be8d0020a None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:06.614210 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d81ba0-909d-40f6-966f-fd375863bf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:06.615053 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d81ba0-909d-40f6-966f-fd375863bf28 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:06.616018 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d81ba0-909d-40f6-966f-fd375863bf28 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:06.616648 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d81ba0-909d-40f6-966f-fd375863bf28 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:06.617782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d81ba0-909d-40f6-966f-fd375863bf28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:06.618676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d81ba0-909d-40f6-966f-fd375863bf28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:06.619269 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:06.620534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d81ba0-909d-40f6-966f-fd375863bf28 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:06.640270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 400/806] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:06.640270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d81ba0-909d-40f6-966f-fd375863bf28 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:06.706877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49443036-57d9-46e4-bf12-ff3be8d0020a None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:06.746539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e0a5b-ba7a-4f35-9007-584647e3a121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:06.747177 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e0a5b-ba7a-4f35-9007-584647e3a121 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:06.747464 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e0a5b-ba7a-4f35-9007-584647e3a121 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:06.747739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e0a5b-ba7a-4f35-9007-584647e3a121 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:06.748393 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e0a5b-ba7a-4f35-9007-584647e3a121 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:06.748695 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e0a5b-ba7a-4f35-9007-584647e3a121 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:06.750055 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e0a5b-ba7a-4f35-9007-584647e3a121 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:06.751697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d81ba0-909d-40f6-966f-fd375863bf28 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:06.758289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 407/807] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:15:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:06.759577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913404e-1ab9-49fe-98fe-f2dbf6c20f44 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:06.759862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913404e-1ab9-49fe-98fe-f2dbf6c20f44 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:06.760137 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913404e-1ab9-49fe-98fe-f2dbf6c20f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:06.777930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-1913404e-1ab9-49fe-98fe-f2dbf6c20f44 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:06.779132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-1913404e-1ab9-49fe-98fe-f2dbf6c20f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:06.793615 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df2346-c99e-42e7-9c56-26ace3a4043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:06.794368 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df2346-c99e-42e7-9c56-26ace3a4043e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:06.794653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df2346-c99e-42e7-9c56-26ace3a4043e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:06.795012 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df2346-c99e-42e7-9c56-26ace3a4043e None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:06.795679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df2346-c99e-42e7-9c56-26ace3a4043e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:06.795985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df2346-c99e-42e7-9c56-26ace3a4043e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:06.797360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df2346-c99e-42e7-9c56-26ace3a4043e None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:06.814604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 401/808] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:06.814604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df2346-c99e-42e7-9c56-26ace3a4043e None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:06.841074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:06.854294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 408/809] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:06.860971 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913404e-1ab9-49fe-98fe-f2dbf6c20f44 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:06.871264 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:06.916891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df2346-c99e-42e7-9c56-26ace3a4043e None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:06.955035 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6235bf-102b-48fa-8f19-fb2d17dab971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:06.956131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6235bf-102b-48fa-8f19-fb2d17dab971 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:06.956414 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6235bf-102b-48fa-8f19-fb2d17dab971 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:06.956684 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6235bf-102b-48fa-8f19-fb2d17dab971 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:06.961611 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913404e-1ab9-49fe-98fe-f2dbf6c20f44 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:06.963875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6235bf-102b-48fa-8f19-fb2d17dab971 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:06.965313 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6235bf-102b-48fa-8f19-fb2d17dab971 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:06.976892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-3c6235bf-102b-48fa-8f19-fb2d17dab971 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:06.976892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:06.976892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:06.976892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:06.976892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:06.976892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:06.976892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:06.976892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:06.976892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:06.976892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:06.976892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:06.976892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:06.976892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:15:06.976892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:15:06.976892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:15:06.977824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:15:06.977824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:06.977824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:06.977824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:06.977824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:06.977824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:06.977824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:06.977824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:06.977824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:06.977824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:06.977824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:06.977824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:06.977824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:06.977824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:06.977824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:06.977824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:06.977824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:06.978781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:15:06.978781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:15:06.978781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:15:06.978781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:15:06.978781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:15:06.978781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:15:06.978781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:06.978781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:06.978781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 402/810] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:06.978781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6235bf-102b-48fa-8f19-fb2d17dab971 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:07.001022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754ee66-4df9-47e8-9db4-393435419e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:07.002118 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754ee66-4df9-47e8-9db4-393435419e2b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:07.002342 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754ee66-4df9-47e8-9db4-393435419e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:07.002760 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754ee66-4df9-47e8-9db4-393435419e2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:07.013984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754ee66-4df9-47e8-9db4-393435419e2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:07.015470 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754ee66-4df9-47e8-9db4-393435419e2b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:07.030241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-7754ee66-4df9-47e8-9db4-393435419e2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:07.030241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:07.030241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:07.030241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:07.030241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:07.030241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:07.030241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:07.030241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:07.030241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:07.030241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:07.030241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:07.030241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:07.030241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:07.030241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:07.030241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:07.030241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:07.031496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:07.031496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:07.031496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:07.031496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:07.031496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:07.031496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:07.031496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:07.031496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:07.031496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:07.031496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:07.031496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:07.031496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:07.031496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:07.031496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:07.031496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:07.031496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:07.031496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:07.032804 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:07.032804 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:07.032804 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:07.032804 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 409/811] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:07.032804 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754ee66-4df9-47e8-9db4-393435419e2b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:07.069799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:07.088893 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6235bf-102b-48fa-8f19-fb2d17dab971 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:07.094460 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:07.127978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61a3bd-6830-4ce4-9e49-c3e79377deb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:07.128611 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61a3bd-6830-4ce4-9e49-c3e79377deb1 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:07.128893 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61a3bd-6830-4ce4-9e49-c3e79377deb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:07.129168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61a3bd-6830-4ce4-9e49-c3e79377deb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:07.131293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754ee66-4df9-47e8-9db4-393435419e2b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:07.136211 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61a3bd-6830-4ce4-9e49-c3e79377deb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:07.137654 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61a3bd-6830-4ce4-9e49-c3e79377deb1 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:07.148566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-7a61a3bd-6830-4ce4-9e49-c3e79377deb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:07.148566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:07.148566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:07.148566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:07.148566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:07.148566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:07.148566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:07.148566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:07.148566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:07.148566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:07.148566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:07.148566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:07.148566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:15:07.148566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:15:07.148566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:15:07.150170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:15:07.150170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:07.150170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:07.150170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:07.150170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:07.150170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:07.150170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:07.150170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:07.150170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:07.150170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:07.150170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:07.150170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:07.150170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:07.150170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:07.150170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:07.150170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:07.150170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:07.151088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:15:07.151088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:15:07.151088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:15:07.151088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:15:07.151088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:15:07.151088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:15:07.151088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:07.151088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:07.151088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 403/812] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:07.151088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61a3bd-6830-4ce4-9e49-c3e79377deb1 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:07.165889 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa801d5-5197-4229-924e-b67410d7382d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:07.166511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa801d5-5197-4229-924e-b67410d7382d None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:07.166811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa801d5-5197-4229-924e-b67410d7382d None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:07.167081 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa801d5-5197-4229-924e-b67410d7382d None admin] PATH_INFO: `/v3/services/network` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:07.167650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa801d5-5197-4229-924e-b67410d7382d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:07.169004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa801d5-5197-4229-924e-b67410d7382d None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:07.177395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-3fa801d5-5197-4229-924e-b67410d7382d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 14:15:07.177395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:07.177395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:07.177395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:07.177395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:07.177395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:07.177395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:07.177395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:07.177395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 14:15:07.177395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:07.177395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:07.177395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:07.177395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:07.177395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:07.177395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:07.177395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:07.177395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:07.178364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:07.178364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:07.178364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:07.178364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:07.178364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:07.178364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:07.178364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:07.178364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:07.178364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:07.178364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:07.178364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:07.178364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:07.178364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:07.178364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:07.178364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:07.178364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:07.179261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:07.179261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:07.179261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:07.179261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:07.179261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:07.179261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:07.179261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:07.179261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:07.179261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:07.179261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:07.179261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:07.179261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:07.179261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 14:15:07.179261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:07.179261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 410/813] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:07 2022] GET /identity/v3/services/network => generated 88 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:07.180067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa801d5-5197-4229-924e-b67410d7382d None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:07.258187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61a3bd-6830-4ce4-9e49-c3e79377deb1 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:07.275158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa801d5-5197-4229-924e-b67410d7382d None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:07.291517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd81a4-1b3f-4191-aff5-469c14526b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:07.292121 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd81a4-1b3f-4191-aff5-469c14526b0d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:07.292399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd81a4-1b3f-4191-aff5-469c14526b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:07.292670 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd81a4-1b3f-4191-aff5-469c14526b0d None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:07.293350 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd81a4-1b3f-4191-aff5-469c14526b0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:07.293654 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd81a4-1b3f-4191-aff5-469c14526b0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:07.295112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd81a4-1b3f-4191-aff5-469c14526b0d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:07.300314 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:07.307795 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a511c10-5295-42cf-a6b7-ea33f4b47c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:07.307932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 404/814] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:15:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:07.312807 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a511c10-5295-42cf-a6b7-ea33f4b47c59 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:07.312807 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a511c10-5295-42cf-a6b7-ea33f4b47c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:07.313009 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd81a4-1b3f-4191-aff5-469c14526b0d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:07.313099 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a511c10-5295-42cf-a6b7-ea33f4b47c59 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:07.313099 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a511c10-5295-42cf-a6b7-ea33f4b47c59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:07.313099 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a511c10-5295-42cf-a6b7-ea33f4b47c59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:07.313099 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a511c10-5295-42cf-a6b7-ea33f4b47c59 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:07.321828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:07.323874 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 411/815] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:07.324653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a511c10-5295-42cf-a6b7-ea33f4b47c59 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:07.417930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd81a4-1b3f-4191-aff5-469c14526b0d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:07.423304 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a511c10-5295-42cf-a6b7-ea33f4b47c59 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:07.450981 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d97e6a-61c0-4ac3-a5c5-63ca4b56ad2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:07.451612 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d97e6a-61c0-4ac3-a5c5-63ca4b56ad2d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:07.451898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d97e6a-61c0-4ac3-a5c5-63ca4b56ad2d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:07.452171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d97e6a-61c0-4ac3-a5c5-63ca4b56ad2d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:07.452747 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d97e6a-61c0-4ac3-a5c5-63ca4b56ad2d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:07.454159 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d97e6a-61c0-4ac3-a5c5-63ca4b56ad2d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:07.456712 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3059f17-4bcf-4d8f-9ce2-c9d6ff26b5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:07.457307 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3059f17-4bcf-4d8f-9ce2-c9d6ff26b5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:07.457584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3059f17-4bcf-4d8f-9ce2-c9d6ff26b5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:07.457852 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3059f17-4bcf-4d8f-9ce2-c9d6ff26b5a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:07.458526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3059f17-4bcf-4d8f-9ce2-c9d6ff26b5a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:07.458859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3059f17-4bcf-4d8f-9ce2-c9d6ff26b5a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:07.460260 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3059f17-4bcf-4d8f-9ce2-c9d6ff26b5a3 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:07.463248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-d9d97e6a-61c0-4ac3-a5c5-63ca4b56ad2d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 14:15:07.463248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:07.463248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:07.463248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:07.463248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:07.463248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:07.463248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:07.463248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:07.463248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 14:15:07.463248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:07.463248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:07.463248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:07.463248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:07.463248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:07.463248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:07.463248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:07.463248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:07.464419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:07.464419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:07.464419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:07.464419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:07.464419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:07.464419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:07.464419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:07.464419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:07.464419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:07.464419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:07.464419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:07.464419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:07.464419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:07.464419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:07.464419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:07.464419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:07.465644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:07.465644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:07.465644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:07.465644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:07.465644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:07.465644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:07.465644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:07.465644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:07.465644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:07.465644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:07.465644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:07.465644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:07.465644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 14:15:07.465644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:07.465644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 405/816] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Sep 5 14:15:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:07.466465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d97e6a-61c0-4ac3-a5c5-63ca4b56ad2d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:07.473915 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 412/817] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:15:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:07.474702 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3059f17-4bcf-4d8f-9ce2-c9d6ff26b5a3 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:07.521636 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:07.538043 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:07.569498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d97e6a-61c0-4ac3-a5c5-63ca4b56ad2d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:07.573976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3059f17-4bcf-4d8f-9ce2-c9d6ff26b5a3 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:07.603844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447a99b-7d83-4522-ae5a-397c25279673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:07.604457 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447a99b-7d83-4522-ae5a-397c25279673 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:07.604743 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447a99b-7d83-4522-ae5a-397c25279673 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:07.605022 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447a99b-7d83-4522-ae5a-397c25279673 None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:07.605649 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447a99b-7d83-4522-ae5a-397c25279673 None admin] RBAC: Adding query filter params (name=network) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:07.605959 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447a99b-7d83-4522-ae5a-397c25279673 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:07.607379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447a99b-7d83-4522-ae5a-397c25279673 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:07.607751 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390561f-86f3-4b93-8720-844e5cf20c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:07.608344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390561f-86f3-4b93-8720-844e5cf20c0f None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:07.608618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390561f-86f3-4b93-8720-844e5cf20c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:07.609116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390561f-86f3-4b93-8720-844e5cf20c0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:07.609596 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390561f-86f3-4b93-8720-844e5cf20c0f None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=709b9ae82dcd47408fc9a6d54a54428a) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:07.609895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390561f-86f3-4b93-8720-844e5cf20c0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:07.611339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390561f-86f3-4b93-8720-844e5cf20c0f None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:07.611804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 406/818] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 5 14:15:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:07.612577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447a99b-7d83-4522-ae5a-397c25279673 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:07.617166 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 413/819] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:15:07 2022] GET /identity/v3/role_assignments?user.id=709b9ae82dcd47408fc9a6d54a54428a&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:07.617956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390561f-86f3-4b93-8720-844e5cf20c0f None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:07.730869 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390561f-86f3-4b93-8720-844e5cf20c0f None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:07.737828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447a99b-7d83-4522-ae5a-397c25279673 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:07.746221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:07.773748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3ae98-9582-41b9-b6a0-01c5cb195ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:07.774353 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3ae98-9582-41b9-b6a0-01c5cb195ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:07.774645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3ae98-9582-41b9-b6a0-01c5cb195ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:07.774960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3ae98-9582-41b9-b6a0-01c5cb195ef5 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:07.775255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a06fb-173d-4a8c-a9f9-efe73a17dacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:07.775601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3ae98-9582-41b9-b6a0-01c5cb195ef5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:07.775891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a06fb-173d-4a8c-a9f9-efe73a17dacb None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:07.775984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3ae98-9582-41b9-b6a0-01c5cb195ef5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:07.776173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a06fb-173d-4a8c-a9f9-efe73a17dacb None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:07.776444 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a06fb-173d-4a8c-a9f9-efe73a17dacb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:07.777262 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3ae98-9582-41b9-b6a0-01c5cb195ef5 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:07.778420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:07.784158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 414/820] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Sep 5 14:15:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:07.785109 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3ae98-9582-41b9-b6a0-01c5cb195ef5 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:07.785266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a06fb-173d-4a8c-a9f9-efe73a17dacb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:07.786705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a06fb-173d-4a8c-a9f9-efe73a17dacb None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:07.800695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-5a5a06fb-173d-4a8c-a9f9-efe73a17dacb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 14:15:07.800695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:07.800695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:07.800695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:07.800695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:07.800695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:07.800695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:07.800695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:07.800695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:07.800695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:07.800695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:07.800695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:07.800695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:15:07.800695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:15:07.800695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:15:07.800695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:07.802201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:07.803237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:07.803237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:07.803237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:15:07.803237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:15:07.803237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:15:07.803237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:15:07.803237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:15:07.803237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:15:07.803237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 14:15:07.803237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:07.803237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 407/821] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:15:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:07.803237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a06fb-173d-4a8c-a9f9-efe73a17dacb None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:07.896442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3ae98-9582-41b9-b6a0-01c5cb195ef5 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:07.909427 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a06fb-173d-4a8c-a9f9-efe73a17dacb None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:07.933200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e19d0-b538-479b-9ac8-12b79b426bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:07.934133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e19d0-b538-479b-9ac8-12b79b426bda None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:07.934620 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e19d0-b538-479b-9ac8-12b79b426bda None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:07.935136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e19d0-b538-479b-9ac8-12b79b426bda None admin] PATH_INFO: `/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/86300433c724483994f59709a3f015db/roles/0a83e40e2d64470a8489bf548e34dabd` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:07.943955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046461a3-0646-4aca-8632-de4e13306ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:07.944575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046461a3-0646-4aca-8632-de4e13306ade None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:07.944859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046461a3-0646-4aca-8632-de4e13306ade None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:07.945137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046461a3-0646-4aca-8632-de4e13306ade None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:07.945767 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046461a3-0646-4aca-8632-de4e13306ade None admin] RBAC: Adding query filter params (type=network) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:07.946614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046461a3-0646-4aca-8632-de4e13306ade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:07.948022 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046461a3-0646-4aca-8632-de4e13306ade None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:07.951853 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e19d0-b538-479b-9ac8-12b79b426bda None admin] RBAC: Authorizing `identity:create_grant(project_id=d9ca42deaed344eea726fe6e32356efc, user_id=86300433c724483994f59709a3f015db, role_id=0a83e40e2d64470a8489bf548e34dabd)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:07.952942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 408/822] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 5 14:15:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:07.953732 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046461a3-0646-4aca-8632-de4e13306ade None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:07.953843 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e19d0-b538-479b-9ac8-12b79b426bda None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:08.013537 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:08.044724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 415/823] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:15:07 2022] PUT /identity/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/86300433c724483994f59709a3f015db/roles/0a83e40e2d64470a8489bf548e34dabd => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:15:08.045699 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e19d0-b538-479b-9ac8-12b79b426bda None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:08.056209 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046461a3-0646-4aca-8632-de4e13306ade None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:08.110180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa303b-26fd-422f-9bac-afc0aa804d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:08.110899 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa303b-26fd-422f-9bac-afc0aa804d2b None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:08.111231 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa303b-26fd-422f-9bac-afc0aa804d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:08.111648 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa303b-26fd-422f-9bac-afc0aa804d2b None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:08.112383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa303b-26fd-422f-9bac-afc0aa804d2b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:08.112762 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa303b-26fd-422f-9bac-afc0aa804d2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:08.113598 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:08.115621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa303b-26fd-422f-9bac-afc0aa804d2b None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:08.122579 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 409/824] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Sep 5 14:15:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:08.124252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa303b-26fd-422f-9bac-afc0aa804d2b None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:08.195972 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e19d0-b538-479b-9ac8-12b79b426bda None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:08.236725 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e53c2-cf4c-4bce-bdbf-058184d27563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:08.237336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e53c2-cf4c-4bce-bdbf-058184d27563 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:08.237615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e53c2-cf4c-4bce-bdbf-058184d27563 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:08.237885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e53c2-cf4c-4bce-bdbf-058184d27563 None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:08.238606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88e53c2-cf4c-4bce-bdbf-058184d27563 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:08.239175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88e53c2-cf4c-4bce-bdbf-058184d27563 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:08.240423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88e53c2-cf4c-4bce-bdbf-058184d27563 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:08.254332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa303b-26fd-422f-9bac-afc0aa804d2b None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:08.259650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 416/825] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Sep 5 14:15:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:08.260795 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e53c2-cf4c-4bce-bdbf-058184d27563 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:08.280093 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:08.292469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ff23f-2bbb-4436-b8c6-4d8f9d065a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:08.293096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ff23f-2bbb-4436-b8c6-4d8f9d065a22 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:08.294580 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ff23f-2bbb-4436-b8c6-4d8f9d065a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:08.294580 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ff23f-2bbb-4436-b8c6-4d8f9d065a22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:08.294580 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ff23f-2bbb-4436-b8c6-4d8f9d065a22 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:08.295969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ff23f-2bbb-4436-b8c6-4d8f9d065a22 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:08.359702 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:08.375727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e53c2-cf4c-4bce-bdbf-058184d27563 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:08.402888 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 410/826] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:15:08 2022] POST /identity/v3/endpoints => generated 337 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:15:08.403815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ff23f-2bbb-4436-b8c6-4d8f9d065a22 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:08.452728 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381c7ac-a101-4f4a-9aca-4319883e0ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:08.455890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381c7ac-a101-4f4a-9aca-4319883e0ad5 None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:08.456178 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381c7ac-a101-4f4a-9aca-4319883e0ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:08.456461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381c7ac-a101-4f4a-9aca-4319883e0ad5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:08.457214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381c7ac-a101-4f4a-9aca-4319883e0ad5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:08.458646 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381c7ac-a101-4f4a-9aca-4319883e0ad5 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:08.562076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ff23f-2bbb-4436-b8c6-4d8f9d065a22 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:08.571809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 417/827] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:15:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:15:08.620824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525befb2-882a-464b-80ee-64243d4136ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:08.621900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525befb2-882a-464b-80ee-64243d4136ef None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:08.622383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525befb2-882a-464b-80ee-64243d4136ef None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:08.622900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525befb2-882a-464b-80ee-64243d4136ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:08.636748 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525befb2-882a-464b-80ee-64243d4136ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:08.638689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525befb2-882a-464b-80ee-64243d4136ef None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:08.655142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-525befb2-882a-464b-80ee-64243d4136ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:08.655142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:08.655142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:08.655142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:08.655142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:08.655142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:08.655142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:08.655142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:08.655142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:08.655142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:08.655142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:08.655142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:08.655142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:08.655142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:08.655142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:08.655142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:08.656140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:08.656140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:08.656140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:08.656140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:08.656140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:08.656140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:08.656140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:08.656140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:08.656140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:08.656140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:08.656140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:08.656140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:08.656140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:08.656140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:08.656140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:08.656140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:08.656140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:08.657031 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:08.657031 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:08.657031 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:08.658901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 411/828] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:08.666830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381c7ac-a101-4f4a-9aca-4319883e0ad5 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:08.785835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381c7ac-a101-4f4a-9aca-4319883e0ad5 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:08.823506 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db865474-38bc-4cc3-8c8e-f7fc3710db0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:08.824048 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db865474-38bc-4cc3-8c8e-f7fc3710db0a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:08.824334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db865474-38bc-4cc3-8c8e-f7fc3710db0a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:08.824607 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db865474-38bc-4cc3-8c8e-f7fc3710db0a None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:08.825253 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db865474-38bc-4cc3-8c8e-f7fc3710db0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:08.825557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db865474-38bc-4cc3-8c8e-f7fc3710db0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:08.826955 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db865474-38bc-4cc3-8c8e-f7fc3710db0a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:08.829383 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:08.839902 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 418/829] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:08.845479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525befb2-882a-464b-80ee-64243d4136ef None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:08.886632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:08.971401 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525befb2-882a-464b-80ee-64243d4136ef None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:09.005843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f24216-7bcb-457e-859a-365be4d9b22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:09.006613 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f24216-7bcb-457e-859a-365be4d9b22b None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:09.006920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f24216-7bcb-457e-859a-365be4d9b22b None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:09.007196 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f24216-7bcb-457e-859a-365be4d9b22b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:09.014644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f24216-7bcb-457e-859a-365be4d9b22b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:09.016162 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f24216-7bcb-457e-859a-365be4d9b22b None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:09.027925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-07f24216-7bcb-457e-859a-365be4d9b22b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:09.027925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:09.027925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:09.027925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:09.027925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:09.027925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:09.027925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:09.027925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:09.027925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:09.027925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:09.027925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:09.027925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:09.027925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:15:09.027925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:15:09.027925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:15:09.028833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:15:09.028833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:09.028833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:09.028833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:09.028833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:09.028833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:09.028833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:09.028833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:09.028833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:09.028833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:09.028833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:09.028833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:09.028833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:09.028833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:09.028833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:09.028833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:09.028833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:09.029763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:15:09.029763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:15:09.029763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:15:09.029763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:15:09.029763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:15:09.029763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:15:09.029763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:09.029763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:09.029763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 412/830] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:09.037582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db865474-38bc-4cc3-8c8e-f7fc3710db0a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:09.145740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db865474-38bc-4cc3-8c8e-f7fc3710db0a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:09.181653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da371c-68d6-46ec-b0e4-534094239b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:09.182268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da371c-68d6-46ec-b0e4-534094239b68 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:09.182549 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da371c-68d6-46ec-b0e4-534094239b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:09.183009 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da371c-68d6-46ec-b0e4-534094239b68 None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:09.183702 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da371c-68d6-46ec-b0e4-534094239b68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:09.184074 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da371c-68d6-46ec-b0e4-534094239b68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:09.185504 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da371c-68d6-46ec-b0e4-534094239b68 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:09.200435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 419/831] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:15:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:09.206324 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f24216-7bcb-457e-859a-365be4d9b22b None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:09.299807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:09.316741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f24216-7bcb-457e-859a-365be4d9b22b None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:09.353338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45a3f8-3190-48e1-af05-e4c0931abdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:09.354339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45a3f8-3190-48e1-af05-e4c0931abdf3 None admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:09.354858 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45a3f8-3190-48e1-af05-e4c0931abdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:09.355343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45a3f8-3190-48e1-af05-e4c0931abdf3 None admin] PATH_INFO: `/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/0a85809d14cb449d848f22a3f39bf641/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:09.357490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45a3f8-3190-48e1-af05-e4c0931abdf3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9ca42deaed344eea726fe6e32356efc, user_id=0a85809d14cb449d848f22a3f39bf641, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:09.359170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45a3f8-3190-48e1-af05-e4c0931abdf3 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:09.439230 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 413/832] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:15:09 2022] PUT /identity/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/0a85809d14cb449d848f22a3f39bf641/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:15:09.657603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46df323-2218-4cec-8b7b-a308fa7e8713 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:09.658090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46df323-2218-4cec-8b7b-a308fa7e8713 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:09.658090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46df323-2218-4cec-8b7b-a308fa7e8713 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:09.659712 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 420/833] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:09.666443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cb879-5b14-4d50-8e20-797429987a73 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:09.666731 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cb879-5b14-4d50-8e20-797429987a73 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:09.667041 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cb879-5b14-4d50-8e20-797429987a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:09.686399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-176cb879-5b14-4d50-8e20-797429987a73 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:09.687606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-176cb879-5b14-4d50-8e20-797429987a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:09.757506 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 414/834] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:10.027396 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46df323-2218-4cec-8b7b-a308fa7e8713 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:10.078076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d485332-c4b8-41a8-b80f-1188af0cbe87 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:10.078605 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d485332-c4b8-41a8-b80f-1188af0cbe87 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:10.078605 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d485332-c4b8-41a8-b80f-1188af0cbe87 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:10.080533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 415/835] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:10.085530 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bae28-8869-4280-b964-3da452093f61 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:10.085831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bae28-8869-4280-b964-3da452093f61 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:10.086107 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bae28-8869-4280-b964-3da452093f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:10.091886 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:10.115253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-ce4bae28-8869-4280-b964-3da452093f61 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:10.116562 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ce4bae28-8869-4280-b964-3da452093f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:10.147555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46df323-2218-4cec-8b7b-a308fa7e8713 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:10.184605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25022e5-41e5-4760-b273-b7876b73d546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:10.185343 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25022e5-41e5-4760-b273-b7876b73d546 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:10.185711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25022e5-41e5-4760-b273-b7876b73d546 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:10.185996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25022e5-41e5-4760-b273-b7876b73d546 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:10.186766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25022e5-41e5-4760-b273-b7876b73d546 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:10.188285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25022e5-41e5-4760-b273-b7876b73d546 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:10.197607 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 416/836] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:10.199201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-e25022e5-41e5-4760-b273-b7876b73d546 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 14:15:10.199201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:10.199201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:10.199201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:10.199201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:10.199201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:10.199201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:10.199201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:10.199201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 14:15:10.199201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:10.199201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:10.199201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:10.199201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:10.199201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:10.199201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:10.199201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:10.200127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:10.200127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:10.200127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:10.200127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:10.200127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:10.200127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:10.200127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:10.200127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:10.200127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:10.200127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:10.200127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:10.200127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:10.200127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:10.200127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:10.200127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:10.200127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:10.200127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:10.201038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:10.201038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:10.201038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:10.201038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:10.201038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:10.201038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:10.201038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:10.201038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:10.201038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:10.201038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:10.201038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:10.201038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:10.201038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 14:15:10.201038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:10.205050 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 421/837] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Sep 5 14:15:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:10.207767 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4bae28-8869-4280-b964-3da452093f61 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:10.216389 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:10.310019 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4bae28-8869-4280-b964-3da452093f61 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:10.349653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee020c-9f70-4dad-81bd-802dbaaf062b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:10.350261 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee020c-9f70-4dad-81bd-802dbaaf062b None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:10.350541 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee020c-9f70-4dad-81bd-802dbaaf062b None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:10.350842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee020c-9f70-4dad-81bd-802dbaaf062b None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:10.351468 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee020c-9f70-4dad-81bd-802dbaaf062b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:10.351771 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee020c-9f70-4dad-81bd-802dbaaf062b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:10.353257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee020c-9f70-4dad-81bd-802dbaaf062b None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:10.358181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 417/838] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 5 14:15:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:10.363605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25022e5-41e5-4760-b273-b7876b73d546 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:10.470070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25022e5-41e5-4760-b273-b7876b73d546 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:10.473567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee020c-9f70-4dad-81bd-802dbaaf062b None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:10.492278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:10.508054 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c27f7-e039-472a-9b02-e35f3eecdc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:10.508917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c27f7-e039-472a-9b02-e35f3eecdc39 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:10.509225 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c27f7-e039-472a-9b02-e35f3eecdc39 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:10.509507 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c27f7-e039-472a-9b02-e35f3eecdc39 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:10.510235 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c27f7-e039-472a-9b02-e35f3eecdc39 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:10.510629 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c27f7-e039-472a-9b02-e35f3eecdc39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:10.512134 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c27f7-e039-472a-9b02-e35f3eecdc39 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:10.517215 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 422/839] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 5 14:15:10 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:10.553643 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:10.591675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee020c-9f70-4dad-81bd-802dbaaf062b None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:10.629347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5211f9b-a0f8-4d08-9a6c-c492f5b38680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:10.630004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5211f9b-a0f8-4d08-9a6c-c492f5b38680 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:10.630289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5211f9b-a0f8-4d08-9a6c-c492f5b38680 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:10.630574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5211f9b-a0f8-4d08-9a6c-c492f5b38680 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:10.642517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5211f9b-a0f8-4d08-9a6c-c492f5b38680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:15:10.648084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-a5211f9b-a0f8-4d08-9a6c-c492f5b38680 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:15:10.648084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:10.648084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:10.648084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:10.648084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:10.648084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:10.648084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:10.648084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:10.648084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:10.648084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:10.648084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:10.648084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:10.648084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:15:10.648084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:15:10.648084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:15:10.648084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:15:10.649079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:15:10.649079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:15:10.649079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:15:10.649079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:15:10.649079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:10.649079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:10.649079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:10.649079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:10.649079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:10.649079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:10.649079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:10.649079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:10.649079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:10.649079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:10.649079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:10.649079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:10.650485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:10.650485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:10.650485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:10.650485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:10.650485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:15:10.650485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:15:10.650485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:15:10.650485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:15:10.650485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:15:10.650485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:15:10.650485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:15:10.650485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:10.650485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 418/840] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:15:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:10.657730 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c27f7-e039-472a-9b02-e35f3eecdc39 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:10.763211 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b794b14-cc58-4c35-bce9-16744a7e0bb3 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:10.763502 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b794b14-cc58-4c35-bce9-16744a7e0bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:10.763782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b794b14-cc58-4c35-bce9-16744a7e0bb3 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:10.765365 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 419/841] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:10.768365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c27f7-e039-472a-9b02-e35f3eecdc39 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:10.770162 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a32e2c-9ce5-4da6-8757-d9e1c9926810 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:10.770450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a32e2c-9ce5-4da6-8757-d9e1c9926810 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:10.770730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a32e2c-9ce5-4da6-8757-d9e1c9926810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:10.788896 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-e2a32e2c-9ce5-4da6-8757-d9e1c9926810 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:10.790626 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e2a32e2c-9ce5-4da6-8757-d9e1c9926810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:10.805181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141d684-40c4-4945-be51-5e0622c05ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:10.805934 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141d684-40c4-4945-be51-5e0622c05ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:10.806201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141d684-40c4-4945-be51-5e0622c05ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:10.806472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141d684-40c4-4945-be51-5e0622c05ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:10.807222 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141d684-40c4-4945-be51-5e0622c05ff6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:10.807534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141d684-40c4-4945-be51-5e0622c05ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:10.809083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141d684-40c4-4945-be51-5e0622c05ff6 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:10.818479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 423/842] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:15:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:10.824260 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141d684-40c4-4945-be51-5e0622c05ff6 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:10.873993 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 420/843] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:10.881628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:10.939616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141d684-40c4-4945-be51-5e0622c05ff6 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:10.974445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06c019-2f22-4978-ba9b-052a37ce40fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:10.975109 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06c019-2f22-4978-ba9b-052a37ce40fd None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:10.975386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06c019-2f22-4978-ba9b-052a37ce40fd None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:10.975653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06c019-2f22-4978-ba9b-052a37ce40fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:10.987043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06c019-2f22-4978-ba9b-052a37ce40fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:10.988517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06c019-2f22-4978-ba9b-052a37ce40fd None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:11.004678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-1d06c019-2f22-4978-ba9b-052a37ce40fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:11.004678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:11.004678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:11.004678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:11.004678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:11.004678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:11.004678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:11.004678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:11.004678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:11.004678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:11.004678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:11.004678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:11.004678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:11.004678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:11.004678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:11.004678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:11.005685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:11.005685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:11.005685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:11.005685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:11.005685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:11.005685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:11.005685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:11.005685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:11.005685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:11.005685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:11.005685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:11.005685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:11.005685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:11.005685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:11.005685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:11.005685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:11.005685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:11.006631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:11.006631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:11.006631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:11.006631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 424/844] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:11.013629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a32e2c-9ce5-4da6-8757-d9e1c9926810 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:11.079162 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:11.124391 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a32e2c-9ce5-4da6-8757-d9e1c9926810 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:11.158676 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06c019-2f22-4978-ba9b-052a37ce40fd None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:11.159725 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2525338-7d10-480d-8dec-691e5462cb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:11.160352 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2525338-7d10-480d-8dec-691e5462cb85 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:11.160636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2525338-7d10-480d-8dec-691e5462cb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:11.160908 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2525338-7d10-480d-8dec-691e5462cb85 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:11.161551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2525338-7d10-480d-8dec-691e5462cb85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:11.161853 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2525338-7d10-480d-8dec-691e5462cb85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:11.163280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2525338-7d10-480d-8dec-691e5462cb85 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:11.176088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 421/845] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:11.182417 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2525338-7d10-480d-8dec-691e5462cb85 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:11.290686 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:11.295382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06c019-2f22-4978-ba9b-052a37ce40fd None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:11.311767 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2525338-7d10-480d-8dec-691e5462cb85 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:11.339819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:11.344980 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d009f26-62c4-4a0a-be7c-7d1dae17beed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:11.345659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d009f26-62c4-4a0a-be7c-7d1dae17beed None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:11.345951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d009f26-62c4-4a0a-be7c-7d1dae17beed None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:11.346311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d009f26-62c4-4a0a-be7c-7d1dae17beed None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:11.347103 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d009f26-62c4-4a0a-be7c-7d1dae17beed None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:11.349127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d009f26-62c4-4a0a-be7c-7d1dae17beed None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:11.356380 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d30ee-4096-44f1-ab96-2941d6f14d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:11.357339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d30ee-4096-44f1-ab96-2941d6f14d65 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:11.357846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d30ee-4096-44f1-ab96-2941d6f14d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:11.358359 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d30ee-4096-44f1-ab96-2941d6f14d65 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:11.360141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-9d009f26-62c4-4a0a-be7c-7d1dae17beed None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 14:15:11.360141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:11.360141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:11.360141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:11.360141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:11.360141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:11.360141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:11.360141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:11.360141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 14:15:11.360141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:11.360141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:11.360141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:11.360141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:11.360141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:11.360141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:11.360141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:11.360141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:11.361117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:11.361117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:11.361117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:11.361117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:11.361117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:11.361117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:11.361117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:11.361117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:11.361117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:11.361117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:11.361117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:11.361117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:11.361117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:11.361117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:11.361117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:11.361117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:11.361951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:11.361951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:11.361951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:11.361951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:11.361951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:11.361951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:11.361951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:11.361951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:11.361951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:11.361951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:11.361951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:11.361951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:11.361951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 14:15:11.361951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:11.361951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 425/846] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:15:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:11.368879 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d009f26-62c4-4a0a-be7c-7d1dae17beed None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:11.371327 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d30ee-4096-44f1-ab96-2941d6f14d65 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:11.372859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d30ee-4096-44f1-ab96-2941d6f14d65 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:11.385738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-783d30ee-4096-44f1-ab96-2941d6f14d65 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 14:15:11.385738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:11.385738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:11.385738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:11.385738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:11.385738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:11.385738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:11.385738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:11.385738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:11.385738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:11.385738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:11.385738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:11.385738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:15:11.385738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:15:11.385738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:15:11.385738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:11.386686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:11.387989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:11.387989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:11.387989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:15:11.387989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:15:11.387989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:15:11.387989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:15:11.387989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:15:11.387989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:15:11.387989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 14:15:11.387989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:11.390934 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 422/847] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 5 14:15:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:11.392908 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d30ee-4096-44f1-ab96-2941d6f14d65 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:11.478939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d009f26-62c4-4a0a-be7c-7d1dae17beed None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:11.503659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d30ee-4096-44f1-ab96-2941d6f14d65 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:11.517851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ed182-e6dd-49f8-8fab-21712835438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:11.519006 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ed182-e6dd-49f8-8fab-21712835438e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:11.519289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ed182-e6dd-49f8-8fab-21712835438e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:11.519848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ed182-e6dd-49f8-8fab-21712835438e None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:11.520498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ed182-e6dd-49f8-8fab-21712835438e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:11.520947 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ed182-e6dd-49f8-8fab-21712835438e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:11.522703 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ed182-e6dd-49f8-8fab-21712835438e None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:11.528233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 426/848] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:15:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:11.529448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7336e5-abeb-463e-835e-e00b1123fe0f None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:11.529737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7336e5-abeb-463e-835e-e00b1123fe0f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:11.530013 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7336e5-abeb-463e-835e-e00b1123fe0f None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:11.531372 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 427/849] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:11.535591 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7336e5-abeb-463e-835e-e00b1123fe0f None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:11.544990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b27ac7-9fdf-4d2d-b9b2-c5c9cb8676d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:11.545599 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b27ac7-9fdf-4d2d-b9b2-c5c9cb8676d3 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:11.545878 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b27ac7-9fdf-4d2d-b9b2-c5c9cb8676d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:11.546152 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b27ac7-9fdf-4d2d-b9b2-c5c9cb8676d3 None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:11.546914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b27ac7-9fdf-4d2d-b9b2-c5c9cb8676d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:11.547214 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b27ac7-9fdf-4d2d-b9b2-c5c9cb8676d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:11.548632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b27ac7-9fdf-4d2d-b9b2-c5c9cb8676d3 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:11.565774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 423/850] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Sep 5 14:15:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:11.567269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296caaa-44f7-40ea-9b90-23dfcc95ee99 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:11.567552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296caaa-44f7-40ea-9b90-23dfcc95ee99 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:11.567822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296caaa-44f7-40ea-9b90-23dfcc95ee99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:11.573547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:11.585951 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-0296caaa-44f7-40ea-9b90-23dfcc95ee99 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:11.588485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-0296caaa-44f7-40ea-9b90-23dfcc95ee99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:11.616684 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:11.656731 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 424/851] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:11.657913 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296caaa-44f7-40ea-9b90-23dfcc95ee99 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:11.663648 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7336e5-abeb-463e-835e-e00b1123fe0f None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:11.700301 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aa376-e87a-4082-83de-7c134b9a72ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:11.700909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aa376-e87a-4082-83de-7c134b9a72ce None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:11.701187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aa376-e87a-4082-83de-7c134b9a72ce None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:11.701455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aa376-e87a-4082-83de-7c134b9a72ce None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:11.702075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aa376-e87a-4082-83de-7c134b9a72ce None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:11.702371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aa376-e87a-4082-83de-7c134b9a72ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:11.703891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aa376-e87a-4082-83de-7c134b9a72ce None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:11.718834 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 428/852] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:15:11 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:11.781166 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296caaa-44f7-40ea-9b90-23dfcc95ee99 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:11.818132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0cf8e-b034-4fce-9798-83d4958d8c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:11.818808 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0cf8e-b034-4fce-9798-83d4958d8c8f None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:11.819096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0cf8e-b034-4fce-9798-83d4958d8c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:11.819365 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0cf8e-b034-4fce-9798-83d4958d8c8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:11.863174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:11.900476 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0cf8e-b034-4fce-9798-83d4958d8c8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:11.901951 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0cf8e-b034-4fce-9798-83d4958d8c8f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:11.935857 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aa376-e87a-4082-83de-7c134b9a72ce None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:12.024335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 425/853] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:15:11 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:15:12.029887 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0cf8e-b034-4fce-9798-83d4958d8c8f None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:12.049800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:12.063866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aa376-e87a-4082-83de-7c134b9a72ce None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:12.108598 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:12.126672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd806de4-678e-4aa4-b7bb-029733f32425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:12.127721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd806de4-678e-4aa4-b7bb-029733f32425 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:12.128523 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd806de4-678e-4aa4-b7bb-029733f32425 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:12.128970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd806de4-678e-4aa4-b7bb-029733f32425 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:12.144011 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd806de4-678e-4aa4-b7bb-029733f32425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:15:12.148258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0cf8e-b034-4fce-9798-83d4958d8c8f None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:12.152385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-fd806de4-678e-4aa4-b7bb-029733f32425 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:15:12.152385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:12.152385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:12.152385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:12.152385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:12.152385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:12.152385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:12.152385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:12.152385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:12.152385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:12.152385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:12.152385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:12.152385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:15:12.152385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:15:12.152385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:15:12.152385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:15:12.153539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:15:12.153539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:15:12.153539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:15:12.153539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:15:12.153539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:12.153539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:12.153539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:12.153539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:12.153539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:12.153539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:12.153539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:12.153539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:12.153539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:12.153539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:12.153539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:12.153539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:12.154582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:12.154582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:12.154582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:12.154582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:12.154582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:15:12.154582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:15:12.154582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:15:12.154582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:15:12.154582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:15:12.154582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:15:12.154582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:15:12.154582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:12.161965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 429/854] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:15:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:12.163143 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd806de4-678e-4aa4-b7bb-029733f32425 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:12.193266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d7de1-94ea-4092-a03f-a7d5032473b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:12.193785 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d7de1-94ea-4092-a03f-a7d5032473b6 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:12.194224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d7de1-94ea-4092-a03f-a7d5032473b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:12.194601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d7de1-94ea-4092-a03f-a7d5032473b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:12.207238 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d7de1-94ea-4092-a03f-a7d5032473b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:12.209125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d7de1-94ea-4092-a03f-a7d5032473b6 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:12.226565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-529d7de1-94ea-4092-a03f-a7d5032473b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:12.226565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:12.226565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:12.226565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:12.226565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:12.226565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:12.226565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:12.226565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:12.226565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:12.226565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:12.226565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:12.226565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:12.226565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:12.226565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:12.226565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:12.226565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:12.228353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:12.228353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:12.228353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:12.228353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:12.228353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:12.228353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:12.228353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:12.228353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:12.228353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:12.228353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:12.228353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:12.228353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:12.228353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:12.228353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:12.228353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:12.228353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:12.228353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:12.229490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:12.229490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:12.229490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:12.235250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 426/855] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:12.239938 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d7de1-94ea-4092-a03f-a7d5032473b6 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:12.294689 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd806de4-678e-4aa4-b7bb-029733f32425 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:12.331018 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65188d-1145-4b37-ae1d-ddebacbb2da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:12.332031 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65188d-1145-4b37-ae1d-ddebacbb2da6 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:12.332204 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65188d-1145-4b37-ae1d-ddebacbb2da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:12.332577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65188d-1145-4b37-ae1d-ddebacbb2da6 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:12.333344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65188d-1145-4b37-ae1d-ddebacbb2da6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:12.333766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65188d-1145-4b37-ae1d-ddebacbb2da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:12.335463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65188d-1145-4b37-ae1d-ddebacbb2da6 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:12.349611 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 430/856] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:15:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:12.350680 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65188d-1145-4b37-ae1d-ddebacbb2da6 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:12.371737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:12.371915 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d7de1-94ea-4092-a03f-a7d5032473b6 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:12.395415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:12.413706 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229a111-6506-4f8d-b7f7-d5161bcaf75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:12.415129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229a111-6506-4f8d-b7f7-d5161bcaf75b None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:12.415780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229a111-6506-4f8d-b7f7-d5161bcaf75b None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:12.416266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229a111-6506-4f8d-b7f7-d5161bcaf75b None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:12.417270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229a111-6506-4f8d-b7f7-d5161bcaf75b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:12.418116 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229a111-6506-4f8d-b7f7-d5161bcaf75b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:12.419745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229a111-6506-4f8d-b7f7-d5161bcaf75b None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:12.436296 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 427/857] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:12.442400 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229a111-6506-4f8d-b7f7-d5161bcaf75b None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:12.475468 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65188d-1145-4b37-ae1d-ddebacbb2da6 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:12.511632 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d745af-832d-456f-862d-2fca9c252a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:12.512263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d745af-832d-456f-862d-2fca9c252a9f None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:12.512542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d745af-832d-456f-862d-2fca9c252a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:12.512819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d745af-832d-456f-862d-2fca9c252a9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:12.523782 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d745af-832d-456f-862d-2fca9c252a9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:12.525807 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d745af-832d-456f-862d-2fca9c252a9f None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:12.541067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-a1d745af-832d-456f-862d-2fca9c252a9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:12.541067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:12.541067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:12.541067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:12.541067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:12.541067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:12.541067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:12.541067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:12.541067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:12.541067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:12.541067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:12.541067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:12.541067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:12.541067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:12.541067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:12.541067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:12.542035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:12.542035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:12.542035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:12.542035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:12.542035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:12.542035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:12.542035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:12.542035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:12.542035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:12.542035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:12.542035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:12.542035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:12.542035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:12.542035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:12.542035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:12.542035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:12.542035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:12.542980 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:12.542980 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:12.542980 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:12.542980 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 431/858] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:12.543849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b990a-edd2-4c06-8227-e97af7f20a06 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:12.543849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b990a-edd2-4c06-8227-e97af7f20a06 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:12.543849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b990a-edd2-4c06-8227-e97af7f20a06 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:12.545199 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 432/859] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:12.549798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b990a-edd2-4c06-8227-e97af7f20a06 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:12.555982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229a111-6506-4f8d-b7f7-d5161bcaf75b None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:12.592190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df1b43-3053-4782-8e71-545a7cc3e707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:12.592841 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df1b43-3053-4782-8e71-545a7cc3e707 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:12.593127 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df1b43-3053-4782-8e71-545a7cc3e707 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:12.593402 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df1b43-3053-4782-8e71-545a7cc3e707 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:12.601130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df1b43-3053-4782-8e71-545a7cc3e707 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:12.603143 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df1b43-3053-4782-8e71-545a7cc3e707 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:12.614570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-40df1b43-3053-4782-8e71-545a7cc3e707 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:12.614570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:12.614570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:12.614570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:12.614570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:12.614570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:12.614570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:12.614570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:12.614570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:12.614570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:12.614570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:12.614570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:12.614570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:15:12.614570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:15:12.614570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:15:12.615505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:15:12.615505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:12.615505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:12.615505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:12.615505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:12.615505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:12.615505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:12.615505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:12.615505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:12.615505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:12.615505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:12.615505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:12.615505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:12.615505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:12.615505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:12.615505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:12.615505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:12.616420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:15:12.616420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:15:12.616420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:15:12.616420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:15:12.616420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:15:12.616420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:15:12.616420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:12.616420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:12.616895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:12.616972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 428/860] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:12.617375 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144552a-bc21-4b72-9dc2-70ff0e8e1892 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:12.617532 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144552a-bc21-4b72-9dc2-70ff0e8e1892 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:12.617814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144552a-bc21-4b72-9dc2-70ff0e8e1892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:12.637584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-7144552a-bc21-4b72-9dc2-70ff0e8e1892 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:12.638806 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-7144552a-bc21-4b72-9dc2-70ff0e8e1892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:12.663204 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b990a-edd2-4c06-8227-e97af7f20a06 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:12.678275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:12.703175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4e70b-5405-4d0c-8e71-1d8d9409e655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:12.704394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4e70b-5405-4d0c-8e71-1d8d9409e655 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:12.704730 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4e70b-5405-4d0c-8e71-1d8d9409e655 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:12.705044 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4e70b-5405-4d0c-8e71-1d8d9409e655 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:12.705716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4e70b-5405-4d0c-8e71-1d8d9409e655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:12.706291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4e70b-5405-4d0c-8e71-1d8d9409e655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:12.707613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4e70b-5405-4d0c-8e71-1d8d9409e655 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:12.720396 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 429/861] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:12.720396 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144552a-bc21-4b72-9dc2-70ff0e8e1892 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:12.720665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 433/862] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:12.731555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4e70b-5405-4d0c-8e71-1d8d9409e655 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:12.824622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144552a-bc21-4b72-9dc2-70ff0e8e1892 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:12.848481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4e70b-5405-4d0c-8e71-1d8d9409e655 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:12.861662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cb5c5-ccb9-4bec-ae0d-16baea76bda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:12.862338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cb5c5-ccb9-4bec-ae0d-16baea76bda2 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:12.862543 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cb5c5-ccb9-4bec-ae0d-16baea76bda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:12.868487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cb5c5-ccb9-4bec-ae0d-16baea76bda2 None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:12.868487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cb5c5-ccb9-4bec-ae0d-16baea76bda2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:12.868487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cb5c5-ccb9-4bec-ae0d-16baea76bda2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:12.868487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cb5c5-ccb9-4bec-ae0d-16baea76bda2 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:12.876565 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:12.883744 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 430/863] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:15:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:12.884792 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cb5c5-ccb9-4bec-ae0d-16baea76bda2 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:12.894737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14675b-20f8-4216-9d62-8e9d4b3ab8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:12.895408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14675b-20f8-4216-9d62-8e9d4b3ab8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:12.895686 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14675b-20f8-4216-9d62-8e9d4b3ab8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:12.899749 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14675b-20f8-4216-9d62-8e9d4b3ab8e7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:12.905881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14675b-20f8-4216-9d62-8e9d4b3ab8e7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:12.907866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14675b-20f8-4216-9d62-8e9d4b3ab8e7 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:12.923849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-3a14675b-20f8-4216-9d62-8e9d4b3ab8e7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 14:15:12.923849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:12.923849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:12.923849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:12.923849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:12.923849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:12.923849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:12.923849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:12.923849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:12.923849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:12.923849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:12.923849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:12.923849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:15:12.923849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:15:12.923849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:15:12.923849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:12.924832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:12.925798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:12.925798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:12.925798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:15:12.925798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:15:12.925798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:15:12.925798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:15:12.925798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:15:12.925798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:15:12.925798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 14:15:12.925798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:12.926355 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:12.929033 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 434/864] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:15:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:12.934509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14675b-20f8-4216-9d62-8e9d4b3ab8e7 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:13.012850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cb5c5-ccb9-4bec-ae0d-16baea76bda2 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:13.046340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb474a-d156-4cdd-9a35-556bdca58a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:13.046975 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb474a-d156-4cdd-9a35-556bdca58a94 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:13.047259 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb474a-d156-4cdd-9a35-556bdca58a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:13.047533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb474a-d156-4cdd-9a35-556bdca58a94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:13.048240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb474a-d156-4cdd-9a35-556bdca58a94 None admin] RBAC: Adding query filter params (role.id=0a83e40e2d64470a8489bf548e34dabd, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=86300433c724483994f59709a3f015db) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:13.048552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb474a-d156-4cdd-9a35-556bdca58a94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:13.049976 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb474a-d156-4cdd-9a35-556bdca58a94 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:13.056096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 431/865] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:15:12 2022] GET /identity/v3/role_assignments?user.id=86300433c724483994f59709a3f015db&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=0a83e40e2d64470a8489bf548e34dabd => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:13.056096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb474a-d156-4cdd-9a35-556bdca58a94 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:13.067212 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14675b-20f8-4216-9d62-8e9d4b3ab8e7 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:13.106396 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26152b9a-1998-4793-b2af-ecff9cc451ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:13.107043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26152b9a-1998-4793-b2af-ecff9cc451ab None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:13.107326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26152b9a-1998-4793-b2af-ecff9cc451ab None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:13.107600 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26152b9a-1998-4793-b2af-ecff9cc451ab None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:13.108340 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26152b9a-1998-4793-b2af-ecff9cc451ab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:13.108643 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26152b9a-1998-4793-b2af-ecff9cc451ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:13.110692 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26152b9a-1998-4793-b2af-ecff9cc451ab None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:13.133008 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 435/866] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Sep 5 14:15:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:13.139366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26152b9a-1998-4793-b2af-ecff9cc451ab None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:13.183912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:13.194988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb474a-d156-4cdd-9a35-556bdca58a94 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:13.204780 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:13.236711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba338afe-41bf-44a0-b8b7-14ec10afb5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:13.237422 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba338afe-41bf-44a0-b8b7-14ec10afb5a4 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:13.237701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba338afe-41bf-44a0-b8b7-14ec10afb5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:13.238078 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba338afe-41bf-44a0-b8b7-14ec10afb5a4 None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:13.238728 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba338afe-41bf-44a0-b8b7-14ec10afb5a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:13.240122 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba338afe-41bf-44a0-b8b7-14ec10afb5a4 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:13.270220 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26152b9a-1998-4793-b2af-ecff9cc451ab None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:13.312431 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4c1ff-93c9-41dd-b4ad-5b6c01bdd9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:13.313051 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4c1ff-93c9-41dd-b4ad-5b6c01bdd9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:13.313329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4c1ff-93c9-41dd-b4ad-5b6c01bdd9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:13.313598 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4c1ff-93c9-41dd-b4ad-5b6c01bdd9f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:13.321356 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 432/867] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 5 14:15:13 2022] POST /identity/v3/services => generated 269 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:15:13.392754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4c1ff-93c9-41dd-b4ad-5b6c01bdd9f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:13.394219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4c1ff-93c9-41dd-b4ad-5b6c01bdd9f1 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:13.441674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:13.501501 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 436/868] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:15:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:15:13.507102 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba338afe-41bf-44a0-b8b7-14ec10afb5a4 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:13.623116 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba338afe-41bf-44a0-b8b7-14ec10afb5a4 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:13.662824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185e4d9-74c9-406d-8de2-fae98fd38112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:13.663772 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185e4d9-74c9-406d-8de2-fae98fd38112 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:13.664262 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185e4d9-74c9-406d-8de2-fae98fd38112 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:13.664747 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185e4d9-74c9-406d-8de2-fae98fd38112 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:13.675899 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185e4d9-74c9-406d-8de2-fae98fd38112 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:13.677604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185e4d9-74c9-406d-8de2-fae98fd38112 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:13.694060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-2185e4d9-74c9-406d-8de2-fae98fd38112 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:13.694060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:13.694060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:13.694060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:13.694060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:13.694060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:13.694060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:13.694060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:13.694060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:13.694060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:13.694060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:13.694060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:13.694060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:13.694060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:13.694060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:13.694060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:13.695173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:13.695173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:13.695173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:13.695173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:13.695173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:13.695173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:13.695173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:13.695173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:13.695173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:13.695173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:13.695173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:13.695173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:13.695173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:13.695173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:13.695173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:13.695173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:13.695173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:13.696678 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:13.696678 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:13.696678 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:13.698498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 433/869] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:13.702872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfcfad-851f-42fa-a102-d23198d75c7e None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:13.702872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfcfad-851f-42fa-a102-d23198d75c7e None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:13.702872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfcfad-851f-42fa-a102-d23198d75c7e None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:13.703380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 437/870] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:13.704872 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185e4d9-74c9-406d-8de2-fae98fd38112 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:13.708138 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01653fb1-0d7b-4676-9f37-a5d4425052d7 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:13.708336 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:13.708446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01653fb1-0d7b-4676-9f37-a5d4425052d7 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:13.708689 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01653fb1-0d7b-4676-9f37-a5d4425052d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:13.726727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-01653fb1-0d7b-4676-9f37-a5d4425052d7 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:13.728062 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-01653fb1-0d7b-4676-9f37-a5d4425052d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:13.814879 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 438/871] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:13 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:13.823394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185e4d9-74c9-406d-8de2-fae98fd38112 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:13.862172 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138b820-87e1-4585-b9f8-62d569576770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:13.862804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138b820-87e1-4585-b9f8-62d569576770 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:13.863096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138b820-87e1-4585-b9f8-62d569576770 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:13.863366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138b820-87e1-4585-b9f8-62d569576770 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:13.864003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138b820-87e1-4585-b9f8-62d569576770 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:13.865723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138b820-87e1-4585-b9f8-62d569576770 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:13.865723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138b820-87e1-4585-b9f8-62d569576770 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:13.879504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 434/872] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:13.885107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01653fb1-0d7b-4676-9f37-a5d4425052d7 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:13.957752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:13.987155 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01653fb1-0d7b-4676-9f37-a5d4425052d7 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:14.023246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430f6b2-86c4-41d1-832f-d2c1ad103faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:14.023878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430f6b2-86c4-41d1-832f-d2c1ad103faa None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:14.024218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430f6b2-86c4-41d1-832f-d2c1ad103faa None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:14.024496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430f6b2-86c4-41d1-832f-d2c1ad103faa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:14.031341 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5430f6b2-86c4-41d1-832f-d2c1ad103faa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:14.032786 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5430f6b2-86c4-41d1-832f-d2c1ad103faa None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:14.044664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-5430f6b2-86c4-41d1-832f-d2c1ad103faa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:14.044664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:14.044664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:14.044664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:14.044664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:14.044664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:14.044664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:14.044664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:14.044664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:14.044664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:14.044664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:14.044664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:14.044664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:15:14.044664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:15:14.044664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:15:14.045587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:15:14.045587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:14.045587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:14.045587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:14.045587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:14.045587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:14.045587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:14.045587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:14.045587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:14.045587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:14.045587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:14.045587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:14.045587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:14.045587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:14.045587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:14.045587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:14.045587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:14.046499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:15:14.046499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:15:14.046499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:15:14.046499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:15:14.046499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:15:14.046499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:15:14.046499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:14.046499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:14.046499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 439/873] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:14.053316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138b820-87e1-4585-b9f8-62d569576770 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:14.096206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430f6b2-86c4-41d1-832f-d2c1ad103faa None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:14.132684 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:14.175380 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138b820-87e1-4585-b9f8-62d569576770 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:14.212807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cef2e-bfc2-40b4-a70f-b354c18575a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:14.214228 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cef2e-bfc2-40b4-a70f-b354c18575a6 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:14.214689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cef2e-bfc2-40b4-a70f-b354c18575a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:14.215194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cef2e-bfc2-40b4-a70f-b354c18575a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:14.216178 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cef2e-bfc2-40b4-a70f-b354c18575a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:14.216711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cef2e-bfc2-40b4-a70f-b354c18575a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:14.218311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cef2e-bfc2-40b4-a70f-b354c18575a6 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:14.222625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430f6b2-86c4-41d1-832f-d2c1ad103faa None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:14.232151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 435/874] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:15:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:14.243349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cef2e-bfc2-40b4-a70f-b354c18575a6 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:14.263887 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d05ba6-51e3-4e06-af7d-03c4c891ee84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:14.264642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d05ba6-51e3-4e06-af7d-03c4c891ee84 None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:14.265062 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d05ba6-51e3-4e06-af7d-03c4c891ee84 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:14.265344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d05ba6-51e3-4e06-af7d-03c4c891ee84 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:14.266010 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d05ba6-51e3-4e06-af7d-03c4c891ee84 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:14.266471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:14.271395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d05ba6-51e3-4e06-af7d-03c4c891ee84 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:14.348463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cef2e-bfc2-40b4-a70f-b354c18575a6 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:14.376487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 440/875] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 5 14:15:14 2022] POST /identity/v3/services => generated 251 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:15:14.386664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:14.404858 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd8dec-842a-4b3c-a640-b60e268cbc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:14.405500 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd8dec-842a-4b3c-a640-b60e268cbc0f None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:14.405780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd8dec-842a-4b3c-a640-b60e268cbc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:14.406111 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd8dec-842a-4b3c-a640-b60e268cbc0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:14.406882 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd8dec-842a-4b3c-a640-b60e268cbc0f None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=0a85809d14cb449d848f22a3f39bf641) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:14.407194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd8dec-842a-4b3c-a640-b60e268cbc0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:14.408890 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd8dec-842a-4b3c-a640-b60e268cbc0f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:14.416302 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 436/876] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:15:14 2022] GET /identity/v3/role_assignments?user.id=0a85809d14cb449d848f22a3f39bf641&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:15.244056 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981882f5-5042-429a-a6bd-85bc5ff8f7a5 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:15.244571 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981882f5-5042-429a-a6bd-85bc5ff8f7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:15.244649 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981882f5-5042-429a-a6bd-85bc5ff8f7a5 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:15.246079 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 441/877] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:15.251358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af5811-e0db-424a-a37e-1acbe1268f08 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:15.252180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af5811-e0db-424a-a37e-1acbe1268f08 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:15.252658 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af5811-e0db-424a-a37e-1acbe1268f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:15.271961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-b5af5811-e0db-424a-a37e-1acbe1268f08 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:15.273441 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b5af5811-e0db-424a-a37e-1acbe1268f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:15.343294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 437/878] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:15.354244 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0b5e2-8325-4c8d-ab48-29770896d9f4 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:15.354480 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0b5e2-8325-4c8d-ab48-29770896d9f4 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:15.354730 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0b5e2-8325-4c8d-ab48-29770896d9f4 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:15.357279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 442/879] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:15.361465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9507f-ccf5-4a30-ae12-21fe9b306560 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:15.362005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9507f-ccf5-4a30-ae12-21fe9b306560 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:15.362472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9507f-ccf5-4a30-ae12-21fe9b306560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:15.382444 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-49f9507f-ccf5-4a30-ae12-21fe9b306560 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:15.384075 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-49f9507f-ccf5-4a30-ae12-21fe9b306560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:15.454289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 438/880] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:15.623043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0b5e2-8325-4c8d-ab48-29770896d9f4 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:15.724636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9507f-ccf5-4a30-ae12-21fe9b306560 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:15.733097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0b5e2-8325-4c8d-ab48-29770896d9f4 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:15.773666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059c915-3862-4ef6-95f5-b9739829bd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:15.774824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059c915-3862-4ef6-95f5-b9739829bd79 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:15.775114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059c915-3862-4ef6-95f5-b9739829bd79 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:15.775389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059c915-3862-4ef6-95f5-b9739829bd79 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:15.779224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:15.791615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059c915-3862-4ef6-95f5-b9739829bd79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:15:15.797904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-7059c915-3862-4ef6-95f5-b9739829bd79 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 14:15:15.797904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:15.797904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:15.797904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:15.797904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:15.797904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:15.797904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:15.797904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:15.797904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:15.797904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:15.797904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:15.797904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:15.797904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:15:15.797904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:15:15.797904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:15:15.797904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:15:15.802348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:15:15.802348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:15:15.802348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:15:15.802348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:15:15.802348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:15.802348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:15.802348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:15.802348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:15.802348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:15.802348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:15.802348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:15.802348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:15.802348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:15.802348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:15.802348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:15.802348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:15.803567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:15.803567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:15.803567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:15.803567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:15.803567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:15:15.803567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:15:15.803567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:15:15.803567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:15:15.803567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:15:15.803567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:15:15.803567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 14:15:15.803567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:15.803567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 443/881] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Sep 5 14:15:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:15.805254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059c915-3862-4ef6-95f5-b9739829bd79 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:15.849260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9507f-ccf5-4a30-ae12-21fe9b306560 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:15.870554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:15.894495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6dcdff-4fbe-48cd-978e-65c695043a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:15.895529 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dcdff-4fbe-48cd-978e-65c695043a1a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:15.896017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dcdff-4fbe-48cd-978e-65c695043a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:15.896493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dcdff-4fbe-48cd-978e-65c695043a1a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:15.897292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6dcdff-4fbe-48cd-978e-65c695043a1a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:15.899543 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6dcdff-4fbe-48cd-978e-65c695043a1a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:15.911594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-5b6dcdff-4fbe-48cd-978e-65c695043a1a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 14:15:15.911594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:15.911594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:15.911594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:15.911594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:15.911594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:15.911594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:15.911594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:15.911594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 14:15:15.911594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:15.911594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:15.911594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:15.911594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:15.911594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:15.911594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:15.911594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:15.917429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:15.917429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:15.917429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:15.917429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:15.917429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:15.917429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:15.917429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:15.917429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:15.917429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:15.917429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:15.917429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:15.917429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:15.917429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:15.917429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:15.917429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:15.917429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:15.917429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:15.918367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:15.918367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:15.918367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:15.918367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:15.918367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:15.918367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:15.918367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:15.918367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:15.918367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:15.918367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:15.918367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:15.918367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:15.918367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 14:15:15.918367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:15.918367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 439/882] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Sep 5 14:15:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:15.920254 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6dcdff-4fbe-48cd-978e-65c695043a1a None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:15.935649 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059c915-3862-4ef6-95f5-b9739829bd79 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:15.972816 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24850d8f-cff8-4f14-8b74-ea1124a4c68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:15.973469 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24850d8f-cff8-4f14-8b74-ea1124a4c68a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:15.973754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24850d8f-cff8-4f14-8b74-ea1124a4c68a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:15.974032 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24850d8f-cff8-4f14-8b74-ea1124a4c68a None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:15.974706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24850d8f-cff8-4f14-8b74-ea1124a4c68a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:15.975042 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24850d8f-cff8-4f14-8b74-ea1124a4c68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:15.976531 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24850d8f-cff8-4f14-8b74-ea1124a4c68a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:15.990640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 444/883] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Sep 5 14:15:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:15.991975 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24850d8f-cff8-4f14-8b74-ea1124a4c68a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:16.058607 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6dcdff-4fbe-48cd-978e-65c695043a1a None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:16.067694 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:16.108180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4ac2d-8801-4b3a-8575-f5f83921c6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:16.108906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24850d8f-cff8-4f14-8b74-ea1124a4c68a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:16.109379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4ac2d-8801-4b3a-8575-f5f83921c6fe None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:16.109896 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4ac2d-8801-4b3a-8575-f5f83921c6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:16.110467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4ac2d-8801-4b3a-8575-f5f83921c6fe None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:16.111499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb4ac2d-8801-4b3a-8575-f5f83921c6fe None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:16.112009 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb4ac2d-8801-4b3a-8575-f5f83921c6fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:16.113613 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb4ac2d-8801-4b3a-8575-f5f83921c6fe None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:16.119839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 440/884] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 5 14:15:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:16.125119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4ac2d-8801-4b3a-8575-f5f83921c6fe None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:16.128524 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:16.150885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d398c-ebc8-4f3f-b92e-983b3aea6632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:16.151454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d398c-ebc8-4f3f-b92e-983b3aea6632 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:16.151732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d398c-ebc8-4f3f-b92e-983b3aea6632 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:16.152002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d398c-ebc8-4f3f-b92e-983b3aea6632 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:16.162678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d398c-ebc8-4f3f-b92e-983b3aea6632 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:16.164379 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d398c-ebc8-4f3f-b92e-983b3aea6632 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:16.179482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-840d398c-ebc8-4f3f-b92e-983b3aea6632 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:16.179482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:16.179482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:16.179482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:16.179482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:16.179482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:16.179482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:16.179482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:16.179482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:16.179482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:16.179482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:16.179482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:16.179482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:16.179482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:16.179482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:16.179482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:16.181218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:16.181218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:16.181218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:16.181218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:16.181218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:16.181218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:16.181218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:16.181218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:16.181218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:16.181218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:16.181218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:16.181218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:16.181218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:16.181218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:16.181218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:16.181218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:16.181218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:16.182251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:16.182251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:16.182251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:16.186246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 445/885] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:16.187287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d398c-ebc8-4f3f-b92e-983b3aea6632 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:16.246145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4ac2d-8801-4b3a-8575-f5f83921c6fe None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:16.285196 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d816c9-5a6e-4b9d-9799-a32a7252f20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:16.286539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d816c9-5a6e-4b9d-9799-a32a7252f20a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:16.287067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d816c9-5a6e-4b9d-9799-a32a7252f20a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:16.287491 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d816c9-5a6e-4b9d-9799-a32a7252f20a None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:16.288290 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d816c9-5a6e-4b9d-9799-a32a7252f20a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:16.288741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d816c9-5a6e-4b9d-9799-a32a7252f20a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:16.289578 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d398c-ebc8-4f3f-b92e-983b3aea6632 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:16.291380 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d816c9-5a6e-4b9d-9799-a32a7252f20a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:16.297651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 441/886] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 5 14:15:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:16.303016 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d816c9-5a6e-4b9d-9799-a32a7252f20a None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:16.330752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca2714-8d76-4eda-8134-ffeeb6bb54b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:16.331379 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca2714-8d76-4eda-8134-ffeeb6bb54b7 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:16.331659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca2714-8d76-4eda-8134-ffeeb6bb54b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:16.331931 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca2714-8d76-4eda-8134-ffeeb6bb54b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:16.332570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca2714-8d76-4eda-8134-ffeeb6bb54b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:16.332868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca2714-8d76-4eda-8134-ffeeb6bb54b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:16.334228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca2714-8d76-4eda-8134-ffeeb6bb54b7 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:16.341469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:16.350995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 446/887] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:16.350995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39f984-9fb3-4e03-a76f-3f8233668a98 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:16.350995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39f984-9fb3-4e03-a76f-3f8233668a98 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:16.350995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39f984-9fb3-4e03-a76f-3f8233668a98 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:16.350995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 447/888] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:16.355475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39f984-9fb3-4e03-a76f-3f8233668a98 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:16.388046 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:16.416924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d816c9-5a6e-4b9d-9799-a32a7252f20a None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:16.455607 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47483298-1e59-4a87-9808-589c8b2cfa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:16.456212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47483298-1e59-4a87-9808-589c8b2cfa38 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:16.456486 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47483298-1e59-4a87-9808-589c8b2cfa38 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:16.456751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47483298-1e59-4a87-9808-589c8b2cfa38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:16.457440 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47483298-1e59-4a87-9808-589c8b2cfa38 None admin] RBAC: Adding query filter params (interface=public, service_id=4c82886b1bc0427bb2a57eea11f776ff, region_id=RegionOne) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:16.457737 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47483298-1e59-4a87-9808-589c8b2cfa38 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:16.459131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47483298-1e59-4a87-9808-589c8b2cfa38 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:16.464633 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 442/889] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Sep 5 14:15:16 2022] GET /identity/v3/endpoints?service_id=4c82886b1bc0427bb2a57eea11f776ff&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:16.464911 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39f984-9fb3-4e03-a76f-3f8233668a98 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:16.465729 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c8518-aeb6-48c8-b1d1-6bf4803b9c82 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:16.466008 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c8518-aeb6-48c8-b1d1-6bf4803b9c82 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:16.466280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c8518-aeb6-48c8-b1d1-6bf4803b9c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:16.483258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-955c8518-aeb6-48c8-b1d1-6bf4803b9c82 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:16.484476 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-955c8518-aeb6-48c8-b1d1-6bf4803b9c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:16.502213 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66a721-c361-44ea-92b1-37170df0e708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:16.502876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66a721-c361-44ea-92b1-37170df0e708 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:16.503162 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66a721-c361-44ea-92b1-37170df0e708 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:16.503433 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66a721-c361-44ea-92b1-37170df0e708 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:16.510840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66a721-c361-44ea-92b1-37170df0e708 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:16.512270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66a721-c361-44ea-92b1-37170df0e708 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:16.523532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-ca66a721-c361-44ea-92b1-37170df0e708 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 14:15:16.523532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:16.523532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:16.523532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:16.523532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:16.523532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:16.523532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:16.523532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:16.523532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:16.523532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:16.523532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:16.523532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:16.523532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:15:16.523532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:15:16.523532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:15:16.523532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:16.525146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:16.526403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:16.526403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:16.526403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:15:16.526403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:15:16.526403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:15:16.526403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:15:16.526403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:15:16.526403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:15:16.526403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 14:15:16.526403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:16.526403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 448/890] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 5 14:15:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:16.526403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c680f-bdb5-4351-abe3-daaf6b593fc0 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:16.527116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c680f-bdb5-4351-abe3-daaf6b593fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:16.527116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c680f-bdb5-4351-abe3-daaf6b593fc0 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:16.531118 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 449/891] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:16.531118 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c680f-bdb5-4351-abe3-daaf6b593fc0 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:16.554144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 443/892] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:16.555002 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c8518-aeb6-48c8-b1d1-6bf4803b9c82 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:16.585097 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:16.616512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:16.633238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c680f-bdb5-4351-abe3-daaf6b593fc0 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:16.662659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c8518-aeb6-48c8-b1d1-6bf4803b9c82 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:16.668920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af9528-81e1-47aa-b336-7969629464c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:16.669524 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af9528-81e1-47aa-b336-7969629464c4 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:16.669805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af9528-81e1-47aa-b336-7969629464c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:16.670101 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af9528-81e1-47aa-b336-7969629464c4 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:16.670700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af9528-81e1-47aa-b336-7969629464c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:16.672086 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af9528-81e1-47aa-b336-7969629464c4 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:16.678109 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 450/893] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 5 14:15:16 2022] GET /identity/v3/services => generated 1268 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:15:16.679562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a94585-5ed9-4429-af35-046de5b9068e None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:16.679923 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a94585-5ed9-4429-af35-046de5b9068e None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:16.680213 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a94585-5ed9-4429-af35-046de5b9068e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:16.700345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-83a94585-5ed9-4429-af35-046de5b9068e None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:16.701606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-83a94585-5ed9-4429-af35-046de5b9068e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:16.722688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fae4b-c57f-49c7-ae45-c97c09502137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:16.723712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fae4b-c57f-49c7-ae45-c97c09502137 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:16.724206 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fae4b-c57f-49c7-ae45-c97c09502137 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:16.724687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fae4b-c57f-49c7-ae45-c97c09502137 None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:16.726421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fae4b-c57f-49c7-ae45-c97c09502137 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:16.727018 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fae4b-c57f-49c7-ae45-c97c09502137 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:16.729315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fae4b-c57f-49c7-ae45-c97c09502137 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:16.759651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 444/894] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Sep 5 14:15:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:16.761600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fae4b-c57f-49c7-ae45-c97c09502137 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:16.786102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 451/895] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:16.836661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a94585-5ed9-4429-af35-046de5b9068e None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:16.860288 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:16.880898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fae4b-c57f-49c7-ae45-c97c09502137 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:16.922778 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746ef87-b108-4b39-be02-5737b942b441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:16.923698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746ef87-b108-4b39-be02-5737b942b441 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:16.924176 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746ef87-b108-4b39-be02-5737b942b441 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:16.924618 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746ef87-b108-4b39-be02-5737b942b441 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:16.932770 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:16.958995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a94585-5ed9-4429-af35-046de5b9068e None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:17.004145 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63863b90-cea3-4a45-95cf-92c1b8c59184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:17.005371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63863b90-cea3-4a45-95cf-92c1b8c59184 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:17.005655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63863b90-cea3-4a45-95cf-92c1b8c59184 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:17.006062 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63863b90-cea3-4a45-95cf-92c1b8c59184 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:17.006806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63863b90-cea3-4a45-95cf-92c1b8c59184 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:17.006973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746ef87-b108-4b39-be02-5737b942b441 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:17.008273 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63863b90-cea3-4a45-95cf-92c1b8c59184 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:17.008740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746ef87-b108-4b39-be02-5737b942b441 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:17.019398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-63863b90-cea3-4a45-95cf-92c1b8c59184 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 14:15:17.019398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:17.019398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:17.019398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:17.019398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:17.019398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:17.019398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:17.019398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:17.019398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 14:15:17.019398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:17.019398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:17.019398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:17.019398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:17.019398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:17.019398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:17.019398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:17.019398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:17.020472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:17.020472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:17.020472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:17.020472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:17.020472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:17.020472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:17.020472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:17.020472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:17.020472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:17.020472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:17.020472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:17.020472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:17.020472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:17.020472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:17.020472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:17.020472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:17.021328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:17.021328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:17.021328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:17.021328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:17.021328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:17.021328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:17.021328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:17.021328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:17.021328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:17.021328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:17.021328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:17.021328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:17.021328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 14:15:17.021328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:17.022095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 452/896] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:15:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:17.028470 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63863b90-cea3-4a45-95cf-92c1b8c59184 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:17.113632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:17.128874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 445/897] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:15:16 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:15:17.131826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746ef87-b108-4b39-be02-5737b942b441 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:17.146499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63863b90-cea3-4a45-95cf-92c1b8c59184 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:17.182884 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae7d39-3d46-4f7a-8e0c-c94ac17aedf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:17.183460 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae7d39-3d46-4f7a-8e0c-c94ac17aedf0 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:17.183739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae7d39-3d46-4f7a-8e0c-c94ac17aedf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:17.184013 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae7d39-3d46-4f7a-8e0c-c94ac17aedf0 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:17.184637 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae7d39-3d46-4f7a-8e0c-c94ac17aedf0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:17.184934 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae7d39-3d46-4f7a-8e0c-c94ac17aedf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:17.186291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae7d39-3d46-4f7a-8e0c-c94ac17aedf0 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:17.191039 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 453/898] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:15:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:17.192132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae7d39-3d46-4f7a-8e0c-c94ac17aedf0 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:17.208210 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:17.239559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746ef87-b108-4b39-be02-5737b942b441 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:17.276717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bae3b2-ca64-4077-b41f-72e938c162e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:17.277504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bae3b2-ca64-4077-b41f-72e938c162e0 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:17.277850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bae3b2-ca64-4077-b41f-72e938c162e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:17.278192 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bae3b2-ca64-4077-b41f-72e938c162e0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:17.278982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bae3b2-ca64-4077-b41f-72e938c162e0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:17.280402 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bae3b2-ca64-4077-b41f-72e938c162e0 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:17.289563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-38bae3b2-ca64-4077-b41f-72e938c162e0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 14:15:17.289563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:17.289563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:17.289563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:17.289563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:17.289563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:17.289563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:17.289563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:17.289563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 14:15:17.289563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:17.289563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:17.289563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:17.289563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:17.289563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:17.289563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:17.289563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:17.289563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:17.291650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:17.291650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:17.291650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:17.291650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:17.291650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:17.291650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:17.291650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:17.291650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:17.291650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:17.291650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:17.291650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:17.291650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:17.291650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:17.291650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:17.291650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:17.291650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:17.292610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:17.292610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:17.292610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:17.292610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:17.292610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:17.292610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:17.292610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:17.292610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:17.292610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:17.292610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:17.292610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:17.292610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:17.292610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 14:15:17.292610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:17.292610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 446/899] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 5 14:15:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:17.293549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bae3b2-ca64-4077-b41f-72e938c162e0 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:17.306991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae7d39-3d46-4f7a-8e0c-c94ac17aedf0 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:17.341822 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ada6c-7574-4627-bd54-9150b8963af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:17.342458 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ada6c-7574-4627-bd54-9150b8963af1 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:17.342811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ada6c-7574-4627-bd54-9150b8963af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:17.343090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ada6c-7574-4627-bd54-9150b8963af1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:17.346303 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:17.354322 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ada6c-7574-4627-bd54-9150b8963af1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:17.359739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ada6c-7574-4627-bd54-9150b8963af1 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:17.369917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-393ada6c-7574-4627-bd54-9150b8963af1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:17.369917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:17.369917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:17.369917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:17.369917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:17.369917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:17.369917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:17.369917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:17.369917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:17.369917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:17.369917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:17.369917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:17.369917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:17.369917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:17.369917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:17.369917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:17.371191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:17.371191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:17.371191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:17.371191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:17.371191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:17.371191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:17.371191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:17.371191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:17.371191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:17.371191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:17.371191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:17.371191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:17.371191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:17.371191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:17.371191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:17.371191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:17.371191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:17.372105 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:17.372105 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:17.372105 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:17.372105 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 454/900] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:17.372346 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ada6c-7574-4627-bd54-9150b8963af1 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:17.402311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bae3b2-ca64-4077-b41f-72e938c162e0 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:17.436280 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:17.439696 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9d4b7-1363-432e-acfb-f211b6772684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:17.440305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9d4b7-1363-432e-acfb-f211b6772684 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:17.440593 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9d4b7-1363-432e-acfb-f211b6772684 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:17.440864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9d4b7-1363-432e-acfb-f211b6772684 None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:17.441759 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9d4b7-1363-432e-acfb-f211b6772684 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:17.442063 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9d4b7-1363-432e-acfb-f211b6772684 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:17.443675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9d4b7-1363-432e-acfb-f211b6772684 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:17.450544 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 447/901] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:15:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:17.450544 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9d4b7-1363-432e-acfb-f211b6772684 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:17.483712 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ada6c-7574-4627-bd54-9150b8963af1 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:17.516902 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3187bc0-83f7-450d-b207-cb9c12d68422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:17.517504 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187bc0-83f7-450d-b207-cb9c12d68422 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:17.517785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187bc0-83f7-450d-b207-cb9c12d68422 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:17.518054 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187bc0-83f7-450d-b207-cb9c12d68422 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:17.518667 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3187bc0-83f7-450d-b207-cb9c12d68422 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:17.518999 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3187bc0-83f7-450d-b207-cb9c12d68422 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:17.520362 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3187bc0-83f7-450d-b207-cb9c12d68422 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:17.525042 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 455/902] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 5 14:15:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:17.530674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3187bc0-83f7-450d-b207-cb9c12d68422 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:17.553673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9d4b7-1363-432e-acfb-f211b6772684 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:17.567935 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:17.592298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d45fb-cc0a-437a-bc48-cbc21be5ecbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:17.592906 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d45fb-cc0a-437a-bc48-cbc21be5ecbc None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:17.593186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d45fb-cc0a-437a-bc48-cbc21be5ecbc None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:17.593459 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d45fb-cc0a-437a-bc48-cbc21be5ecbc None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:17.594091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d45fb-cc0a-437a-bc48-cbc21be5ecbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:17.594392 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d45fb-cc0a-437a-bc48-cbc21be5ecbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:17.595793 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d45fb-cc0a-437a-bc48-cbc21be5ecbc None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:17.608131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 448/903] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:17.608928 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d45fb-cc0a-437a-bc48-cbc21be5ecbc None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:17.641403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3187bc0-83f7-450d-b207-cb9c12d68422 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:17.664764 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:17.675702 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25327993-1c7e-4fa6-a1ce-b5e9a6623f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:17.677398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25327993-1c7e-4fa6-a1ce-b5e9a6623f01 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:17.677398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25327993-1c7e-4fa6-a1ce-b5e9a6623f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:17.677398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25327993-1c7e-4fa6-a1ce-b5e9a6623f01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:17.677650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25327993-1c7e-4fa6-a1ce-b5e9a6623f01 None admin] RBAC: Adding query filter params (interface=public, service_id=d4515ad0f4b44fc8b1732adf4530a669, region_id=RegionOne) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:17.677910 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25327993-1c7e-4fa6-a1ce-b5e9a6623f01 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:17.679312 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25327993-1c7e-4fa6-a1ce-b5e9a6623f01 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:17.684757 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 456/904] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Sep 5 14:15:17 2022] GET /identity/v3/endpoints?service_id=d4515ad0f4b44fc8b1732adf4530a669&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:17.685536 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25327993-1c7e-4fa6-a1ce-b5e9a6623f01 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:17.719260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d45fb-cc0a-437a-bc48-cbc21be5ecbc None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:17.759500 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d11321-6b9f-49b8-8a64-40eea1b40cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:17.760115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d11321-6b9f-49b8-8a64-40eea1b40cfa None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:17.760400 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d11321-6b9f-49b8-8a64-40eea1b40cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:17.760677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d11321-6b9f-49b8-8a64-40eea1b40cfa None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:17.761298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d11321-6b9f-49b8-8a64-40eea1b40cfa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:17.761601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d11321-6b9f-49b8-8a64-40eea1b40cfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:17.762996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d11321-6b9f-49b8-8a64-40eea1b40cfa None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:17.768864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 449/905] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 5 14:15:17 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:17.769863 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d11321-6b9f-49b8-8a64-40eea1b40cfa None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:17.798698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25327993-1c7e-4fa6-a1ce-b5e9a6623f01 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:17.841443 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbaa3b9-5d15-47b5-b237-f486b1837e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:17.842141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaa3b9-5d15-47b5-b237-f486b1837e6e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:17.842427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaa3b9-5d15-47b5-b237-f486b1837e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:17.842703 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaa3b9-5d15-47b5-b237-f486b1837e6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:17.850687 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbaa3b9-5d15-47b5-b237-f486b1837e6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:17.852338 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbaa3b9-5d15-47b5-b237-f486b1837e6e None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:17.853391 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:17.864700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-7cbaa3b9-5d15-47b5-b237-f486b1837e6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:17.864700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:17.864700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:17.864700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:17.864700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:17.864700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:17.864700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:17.864700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:17.864700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:17.864700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:17.864700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:17.864700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:17.864700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:15:17.864700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:15:17.864700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:15:17.865684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:15:17.865684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:17.865684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:17.865684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:17.865684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:17.865684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:17.865684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:17.865684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:17.865684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:17.865684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:17.865684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:17.865684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:17.865684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:17.865684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:17.865684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:17.865684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:17.865684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:17.866618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:15:17.866618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:15:17.866618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:15:17.866618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:15:17.866618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:15:17.866618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:15:17.866618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:17.866618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:17.866618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 457/906] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:17.872214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbaa3b9-5d15-47b5-b237-f486b1837e6e None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:17.918914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d11321-6b9f-49b8-8a64-40eea1b40cfa None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:17.940273 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:17.966870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb88d7-1de3-4813-b65e-7e2c9ddd5b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:17.967548 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb88d7-1de3-4813-b65e-7e2c9ddd5b92 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:17.967835 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb88d7-1de3-4813-b65e-7e2c9ddd5b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:17.968110 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb88d7-1de3-4813-b65e-7e2c9ddd5b92 None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:17.968769 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb88d7-1de3-4813-b65e-7e2c9ddd5b92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:17.970649 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb88d7-1de3-4813-b65e-7e2c9ddd5b92 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:17.979532 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 450/907] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 5 14:15:17 2022] GET /identity/v3/services => generated 1268 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:15:17.997546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbaa3b9-5d15-47b5-b237-f486b1837e6e None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:18.035894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3dae1-1661-494d-8b79-928332101297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:18.036572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3dae1-1661-494d-8b79-928332101297 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:18.036856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3dae1-1661-494d-8b79-928332101297 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:18.037128 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3dae1-1661-494d-8b79-928332101297 None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:18.037912 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3dae1-1661-494d-8b79-928332101297 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:18.038218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3dae1-1661-494d-8b79-928332101297 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:18.040257 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3dae1-1661-494d-8b79-928332101297 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:18.054616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 458/908] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:15:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:18.062974 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb88d7-1de3-4813-b65e-7e2c9ddd5b92 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:18.182953 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb88d7-1de3-4813-b65e-7e2c9ddd5b92 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:18.184731 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:18.219694 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af12fa-e2ed-4945-9c99-def7aa25ae6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:18.220303 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af12fa-e2ed-4945-9c99-def7aa25ae6d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:18.220585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af12fa-e2ed-4945-9c99-def7aa25ae6d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:18.220911 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af12fa-e2ed-4945-9c99-def7aa25ae6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:18.221584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af12fa-e2ed-4945-9c99-def7aa25ae6d None admin] RBAC: Adding query filter params (role.id=48a9bca3f43c4943855875bf97ec564f, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=86300433c724483994f59709a3f015db) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:18.221890 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af12fa-e2ed-4945-9c99-def7aa25ae6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:18.223360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af12fa-e2ed-4945-9c99-def7aa25ae6d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:18.229447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 451/909] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:15:18 2022] GET /identity/v3/role_assignments?user.id=86300433c724483994f59709a3f015db&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=48a9bca3f43c4943855875bf97ec564f => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:18.714665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bad3d2-2a74-404d-af7f-6f096788495d None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:18.715176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bad3d2-2a74-404d-af7f-6f096788495d None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:18.715176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bad3d2-2a74-404d-af7f-6f096788495d None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:18.716729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 459/910] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:18.721714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40f01d-95f7-4c93-8293-70650333792b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:18.722123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40f01d-95f7-4c93-8293-70650333792b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:18.722297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40f01d-95f7-4c93-8293-70650333792b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:18.740950 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-ff40f01d-95f7-4c93-8293-70650333792b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:18.742123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ff40f01d-95f7-4c93-8293-70650333792b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:18.810226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 452/911] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:19.075995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bad3d2-2a74-404d-af7f-6f096788495d None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:19.181173 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bad3d2-2a74-404d-af7f-6f096788495d None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:19.190770 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:19.216219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac121957-3c77-45ac-8a9e-dfa675cc743f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:19.216851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac121957-3c77-45ac-8a9e-dfa675cc743f None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:19.217180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac121957-3c77-45ac-8a9e-dfa675cc743f None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:19.217452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac121957-3c77-45ac-8a9e-dfa675cc743f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:19.218066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac121957-3c77-45ac-8a9e-dfa675cc743f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:19.219448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac121957-3c77-45ac-8a9e-dfa675cc743f None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:19.228827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-ac121957-3c77-45ac-8a9e-dfa675cc743f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 14:15:19.228827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:19.228827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:19.228827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:19.228827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:19.228827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:19.228827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:19.228827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:19.228827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 14:15:19.228827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:19.228827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:19.228827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:19.228827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:19.228827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:19.228827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:19.228827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:19.229800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:19.229800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:19.229800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:19.229800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:19.229800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:19.229800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:19.229800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:19.229800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:19.229800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:19.229800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:19.229800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:19.229800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:19.229800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:19.229800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:19.229800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:19.229800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:19.229800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:19.230950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:19.230950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:19.230950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:19.230950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:19.230950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:19.230950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:19.230950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:19.230950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:19.230950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:19.230950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:19.230950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:19.230950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:19.230950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 14:15:19.230950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:19.230950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 460/912] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Sep 5 14:15:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:19.235986 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40f01d-95f7-4c93-8293-70650333792b None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:19.337552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40f01d-95f7-4c93-8293-70650333792b None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:19.339102 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:19.372678 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3be07d-0f4e-4f0a-81a6-4e19d9ba3706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:19.373319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3be07d-0f4e-4f0a-81a6-4e19d9ba3706 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:19.373637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3be07d-0f4e-4f0a-81a6-4e19d9ba3706 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:19.373908 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3be07d-0f4e-4f0a-81a6-4e19d9ba3706 None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:19.374536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3be07d-0f4e-4f0a-81a6-4e19d9ba3706 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:19.374858 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3be07d-0f4e-4f0a-81a6-4e19d9ba3706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:19.376232 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3be07d-0f4e-4f0a-81a6-4e19d9ba3706 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:19.381213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 453/913] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 5 14:15:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:19.387400 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac121957-3c77-45ac-8a9e-dfa675cc743f None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:19.489280 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac121957-3c77-45ac-8a9e-dfa675cc743f None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:19.522263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faebee9b-67ed-4803-813e-be02d9eba948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:19.522913 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebee9b-67ed-4803-813e-be02d9eba948 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:19.523192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebee9b-67ed-4803-813e-be02d9eba948 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:19.523460 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebee9b-67ed-4803-813e-be02d9eba948 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:19.524635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faebee9b-67ed-4803-813e-be02d9eba948 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:19.524942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faebee9b-67ed-4803-813e-be02d9eba948 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:19.526432 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faebee9b-67ed-4803-813e-be02d9eba948 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:19.531639 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 461/914] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 5 14:15:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:19.537175 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3be07d-0f4e-4f0a-81a6-4e19d9ba3706 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:19.640360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3be07d-0f4e-4f0a-81a6-4e19d9ba3706 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:19.674651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59bf57-dbbd-4425-b8d9-e5d2854b6bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:19.675285 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59bf57-dbbd-4425-b8d9-e5d2854b6bdf None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:19.675563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59bf57-dbbd-4425-b8d9-e5d2854b6bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:19.675840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59bf57-dbbd-4425-b8d9-e5d2854b6bdf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:19.676505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59bf57-dbbd-4425-b8d9-e5d2854b6bdf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:19.677862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59bf57-dbbd-4425-b8d9-e5d2854b6bdf None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:19.749612 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 454/915] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:15:19 2022] POST /identity/v3/endpoints => generated 347 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:15:19.783935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf55bc-cb81-4f5b-b169-5c6bac0fe604 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:19.784930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf55bc-cb81-4f5b-b169-5c6bac0fe604 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:19.785445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf55bc-cb81-4f5b-b169-5c6bac0fe604 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:19.787489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 462/916] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:19.794075 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786a550-79dd-4e7e-96df-eca48b08fe0b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:19.794706 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786a550-79dd-4e7e-96df-eca48b08fe0b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:19.795230 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786a550-79dd-4e7e-96df-eca48b08fe0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:19.815180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-b786a550-79dd-4e7e-96df-eca48b08fe0b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:19.817003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b786a550-79dd-4e7e-96df-eca48b08fe0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:19.829918 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:19.903210 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 455/917] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:19.952259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ba21d-a856-4a80-920f-b30169a158eb None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:19.952586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ba21d-a856-4a80-920f-b30169a158eb None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:19.952808 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ba21d-a856-4a80-920f-b30169a158eb None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:19.954333 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 463/918] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:19.959273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c1202-db1b-4291-83a7-70b0c7012133 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:19.959567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c1202-db1b-4291-83a7-70b0c7012133 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:19.959846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c1202-db1b-4291-83a7-70b0c7012133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:19.979156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-e87c1202-db1b-4291-83a7-70b0c7012133 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:19.980350 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e87c1202-db1b-4291-83a7-70b0c7012133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:20.048466 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 456/919] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:20.187698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ba21d-a856-4a80-920f-b30169a158eb None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:20.224631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:20.225849 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7185d7b-2963-4199-ab52-eb4138128363 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:20.226364 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7185d7b-2963-4199-ab52-eb4138128363 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:20.226867 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7185d7b-2963-4199-ab52-eb4138128363 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:20.229222 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 457/920] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:20.234773 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd052ca-8865-427e-8dd9-03ddad1a694d None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:20.235302 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd052ca-8865-427e-8dd9-03ddad1a694d None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:20.235747 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd052ca-8865-427e-8dd9-03ddad1a694d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:20.254544 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-5cd052ca-8865-427e-8dd9-03ddad1a694d None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:20.260338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-5cd052ca-8865-427e-8dd9-03ddad1a694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:20.303252 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ba21d-a856-4a80-920f-b30169a158eb None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:20.330129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 458/921] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:20.339982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd052ca-8865-427e-8dd9-03ddad1a694d None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:20.342679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:20.347816 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676ad30-076c-452a-b1b4-5ea792335c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:20.348447 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676ad30-076c-452a-b1b4-5ea792335c33 None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:20.348732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676ad30-076c-452a-b1b4-5ea792335c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:20.349007 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676ad30-076c-452a-b1b4-5ea792335c33 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:20.349669 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676ad30-076c-452a-b1b4-5ea792335c33 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:20.351088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676ad30-076c-452a-b1b4-5ea792335c33 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:20.447120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd052ca-8865-427e-8dd9-03ddad1a694d None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:20.448677 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 464/922] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 5 14:15:20 2022] POST /identity/v3/services => generated 241 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:15:20.501117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460a9a4-b49e-4cd6-b425-86e1e1da554f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:20.501841 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460a9a4-b49e-4cd6-b425-86e1e1da554f None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:20.502129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460a9a4-b49e-4cd6-b425-86e1e1da554f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:20.502406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460a9a4-b49e-4cd6-b425-86e1e1da554f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:20.503035 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460a9a4-b49e-4cd6-b425-86e1e1da554f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:20.504610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460a9a4-b49e-4cd6-b425-86e1e1da554f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:20.515340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-b460a9a4-b49e-4cd6-b425-86e1e1da554f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 14:15:20.515340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:20.515340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:20.515340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:20.515340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:20.515340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:20.515340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:20.515340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:20.515340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 14:15:20.515340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:20.515340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:20.515340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:20.515340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:20.515340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:20.515340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:20.515340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:20.515340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:20.516405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:20.516405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:20.516405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:20.516405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:20.516405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:20.516405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:20.516405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:20.516405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:20.516405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:20.516405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:20.516405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:20.516405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:20.516405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:20.516405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:20.516405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:20.516405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:20.517270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:20.517270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:20.517270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:20.517270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:20.517270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:20.517270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:20.517270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:20.517270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:20.517270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:20.517270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:20.517270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:20.517270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:20.517270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 14:15:20.517270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:20.517270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 459/923] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:15:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:20.524883 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676ad30-076c-452a-b1b4-5ea792335c33 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:20.610963 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460a9a4-b49e-4cd6-b425-86e1e1da554f None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:20.629722 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:20.639263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676ad30-076c-452a-b1b4-5ea792335c33 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:20.687319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:20.689316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52874a-f294-4729-9061-1a233c26750c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:20.690543 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52874a-f294-4729-9061-1a233c26750c None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:20.691636 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52874a-f294-4729-9061-1a233c26750c None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:20.692066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52874a-f294-4729-9061-1a233c26750c None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:20.693082 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52874a-f294-4729-9061-1a233c26750c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:20.694028 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52874a-f294-4729-9061-1a233c26750c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:20.696062 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52874a-f294-4729-9061-1a233c26750c None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:20.703104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 465/924] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:15:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:20.708159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52874a-f294-4729-9061-1a233c26750c None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:20.743309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460a9a4-b49e-4cd6-b425-86e1e1da554f None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:20.791894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb812b-b2e1-4b0f-89d1-ad0b05690949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:20.793780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb812b-b2e1-4b0f-89d1-ad0b05690949 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:20.794363 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb812b-b2e1-4b0f-89d1-ad0b05690949 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:20.794810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb812b-b2e1-4b0f-89d1-ad0b05690949 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:20.809468 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb812b-b2e1-4b0f-89d1-ad0b05690949 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:15:20.812592 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52874a-f294-4729-9061-1a233c26750c None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:20.817430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-8ccb812b-b2e1-4b0f-89d1-ad0b05690949 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 14:15:20.817430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:20.817430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:20.817430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:20.817430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:20.817430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:20.817430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:20.817430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:20.817430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:20.817430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:20.817430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:20.817430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:20.817430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:15:20.817430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:15:20.817430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:15:20.817430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:15:20.818587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:15:20.818587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:15:20.818587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:15:20.818587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:15:20.818587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:20.818587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:20.818587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:20.818587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:20.818587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:20.818587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:20.818587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:20.818587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:20.818587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:20.818587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:20.818587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:20.818587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:20.819595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:20.819595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:20.819595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:20.819595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:20.819595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:15:20.819595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:15:20.819595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:15:20.819595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:15:20.819595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:15:20.819595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:15:20.819595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 14:15:20.819595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:20.826868 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 460/925] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Sep 5 14:15:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:20.828521 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb812b-b2e1-4b0f-89d1-ad0b05690949 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:20.854751 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50fd55c-a3a6-4ac4-ab20-4f95956428b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:20.855765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50fd55c-a3a6-4ac4-ab20-4f95956428b0 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:20.856280 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50fd55c-a3a6-4ac4-ab20-4f95956428b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:20.856762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50fd55c-a3a6-4ac4-ab20-4f95956428b0 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:20.857672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50fd55c-a3a6-4ac4-ab20-4f95956428b0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:20.858182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50fd55c-a3a6-4ac4-ab20-4f95956428b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:20.859795 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50fd55c-a3a6-4ac4-ab20-4f95956428b0 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:20.865201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 466/926] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 5 14:15:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:20.873645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50fd55c-a3a6-4ac4-ab20-4f95956428b0 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:20.908962 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:20.968495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb812b-b2e1-4b0f-89d1-ad0b05690949 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:20.985630 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50fd55c-a3a6-4ac4-ab20-4f95956428b0 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:20.995870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:21.008572 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e178082-3400-4e86-ba34-3b7e29f32e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:21.009382 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e178082-3400-4e86-ba34-3b7e29f32e86 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:21.009850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e178082-3400-4e86-ba34-3b7e29f32e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:21.010366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e178082-3400-4e86-ba34-3b7e29f32e86 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:21.011292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e178082-3400-4e86-ba34-3b7e29f32e86 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:21.011760 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e178082-3400-4e86-ba34-3b7e29f32e86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:21.013362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e178082-3400-4e86-ba34-3b7e29f32e86 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:21.022119 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bed200-2298-4834-bffb-c8551d0af807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:21.028475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bed200-2298-4834-bffb-c8551d0af807 None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:21.028475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bed200-2298-4834-bffb-c8551d0af807 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:21.028475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bed200-2298-4834-bffb-c8551d0af807 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:21.028475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bed200-2298-4834-bffb-c8551d0af807 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:21.028475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bed200-2298-4834-bffb-c8551d0af807 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:21.029250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 461/927] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Sep 5 14:15:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:21.030415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e178082-3400-4e86-ba34-3b7e29f32e86 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:21.142553 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e178082-3400-4e86-ba34-3b7e29f32e86 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:21.146942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 467/928] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:15:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:15:21.199708 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5434cb-69ab-4497-b028-b0c7c1357bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:21.200610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5434cb-69ab-4497-b028-b0c7c1357bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:21.201062 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5434cb-69ab-4497-b028-b0c7c1357bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:21.201570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5434cb-69ab-4497-b028-b0c7c1357bb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:21.214253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5434cb-69ab-4497-b028-b0c7c1357bb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:21.215999 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5434cb-69ab-4497-b028-b0c7c1357bb9 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:21.233712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-6a5434cb-69ab-4497-b028-b0c7c1357bb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:21.233712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:21.233712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:21.233712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:21.233712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:21.233712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:21.233712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:21.233712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:21.233712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:21.233712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:21.233712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:21.233712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:21.233712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:21.233712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:21.233712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:21.233712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:21.234925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:21.234925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:21.234925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:21.234925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:21.234925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:21.234925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:21.234925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:21.234925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:21.234925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:21.234925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:21.234925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:21.234925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:21.234925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:21.234925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:21.234925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:21.234925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:21.234925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:21.236031 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:21.236031 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:21.236031 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:21.237985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 462/929] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:21.244107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bed200-2298-4834-bffb-c8551d0af807 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:21.337265 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:21.361214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bed200-2298-4834-bffb-c8551d0af807 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:21.396881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37fb061-3368-4847-8442-b4a3473b4a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:21.397497 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fb061-3368-4847-8442-b4a3473b4a0b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:21.397775 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fb061-3368-4847-8442-b4a3473b4a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:21.398048 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fb061-3368-4847-8442-b4a3473b4a0b None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:21.398753 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37fb061-3368-4847-8442-b4a3473b4a0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:21.399645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37fb061-3368-4847-8442-b4a3473b4a0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:21.401004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37fb061-3368-4847-8442-b4a3473b4a0b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:21.413454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 468/930] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:21.419249 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5434cb-69ab-4497-b028-b0c7c1357bb9 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:21.458467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:21.533107 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5434cb-69ab-4497-b028-b0c7c1357bb9 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:21.569520 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724a55b-852f-4aea-a725-c54afe025a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:21.570510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724a55b-852f-4aea-a725-c54afe025a93 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:21.571143 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724a55b-852f-4aea-a725-c54afe025a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:21.571622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724a55b-852f-4aea-a725-c54afe025a93 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:21.579473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724a55b-852f-4aea-a725-c54afe025a93 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:21.581257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724a55b-852f-4aea-a725-c54afe025a93 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:21.593846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-9724a55b-852f-4aea-a725-c54afe025a93 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 14:15:21.593846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:21.593846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:21.593846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:21.593846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:21.593846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:21.593846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:21.593846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:21.593846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:21.593846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:21.593846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:21.593846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:21.593846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:15:21.593846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:15:21.593846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:15:21.593846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:21.594850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:21.595799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:21.595799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:21.595799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:15:21.595799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:15:21.595799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:15:21.595799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:15:21.595799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:15:21.595799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:15:21.595799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 14:15:21.595799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:21.598284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 463/931] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 5 14:15:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:21.608725 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37fb061-3368-4847-8442-b4a3473b4a0b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:21.772087 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37fb061-3368-4847-8442-b4a3473b4a0b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:21.817040 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa7351-a377-4515-8729-505570b0d520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:21.817647 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa7351-a377-4515-8729-505570b0d520 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:21.817872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa7351-a377-4515-8729-505570b0d520 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:21.820188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa7351-a377-4515-8729-505570b0d520 None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:21.821496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa7351-a377-4515-8729-505570b0d520 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:21.821803 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa7351-a377-4515-8729-505570b0d520 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:21.823814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd3c2f-b925-4657-85f9-a02dd6a2a1cb None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:21.824213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd3c2f-b925-4657-85f9-a02dd6a2a1cb None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:21.824397 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd3c2f-b925-4657-85f9-a02dd6a2a1cb None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:21.826698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa7351-a377-4515-8729-505570b0d520 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:21.829444 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 464/932] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:15:21.832230 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1140259-d0ad-4380-a7b3-2468cbd2ec53 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:21.834811 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1140259-d0ad-4380-a7b3-2468cbd2ec53 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:21.834811 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1140259-d0ad-4380-a7b3-2468cbd2ec53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:21.843185 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:21.854685 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-c1140259-d0ad-4380-a7b3-2468cbd2ec53 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:21.856190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c1140259-d0ad-4380-a7b3-2468cbd2ec53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:21.857001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 469/933] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Sep 5 14:15:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:21.863187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa7351-a377-4515-8729-505570b0d520 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:22.001115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 465/934] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:22.041670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa7351-a377-4515-8729-505570b0d520 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:22.087139 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe12ed-f25a-4d3f-bb33-cd27e8fa4e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:22.088409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe12ed-f25a-4d3f-bb33-cd27e8fa4e0a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:22.088696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe12ed-f25a-4d3f-bb33-cd27e8fa4e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:22.088966 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe12ed-f25a-4d3f-bb33-cd27e8fa4e0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:22.104740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe12ed-f25a-4d3f-bb33-cd27e8fa4e0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:22.106209 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe12ed-f25a-4d3f-bb33-cd27e8fa4e0a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:22.125035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-84fe12ed-f25a-4d3f-bb33-cd27e8fa4e0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:22.125035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:22.125035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:22.125035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:22.125035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:22.125035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:22.125035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:22.125035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:22.125035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:22.125035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:22.125035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:22.125035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:22.125035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:22.125035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:22.125035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:22.125035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:22.126012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:22.126012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:22.126012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:22.126012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:22.126012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:22.126012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:22.126012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:22.126012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:22.126012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:22.126012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:22.126012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:22.126012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:22.126012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:22.126012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:22.126012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:22.126012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:22.126012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:22.126937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:22.126937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:22.126937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:22.133225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1140259-d0ad-4380-a7b3-2468cbd2ec53 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:22.146444 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 470/935] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:22.187136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:22.243154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1140259-d0ad-4380-a7b3-2468cbd2ec53 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:22.285508 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d004c-af24-4bfa-86e9-4acaeb8ad7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:22.286123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d004c-af24-4bfa-86e9-4acaeb8ad7f7 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:22.286400 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d004c-af24-4bfa-86e9-4acaeb8ad7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:22.286668 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d004c-af24-4bfa-86e9-4acaeb8ad7f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:22.289473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d004c-af24-4bfa-86e9-4acaeb8ad7f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:22.289914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d004c-af24-4bfa-86e9-4acaeb8ad7f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:22.291752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d004c-af24-4bfa-86e9-4acaeb8ad7f7 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:22.300279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe12ed-f25a-4d3f-bb33-cd27e8fa4e0a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:22.309606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 466/936] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:22.315214 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d004c-af24-4bfa-86e9-4acaeb8ad7f7 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:22.358730 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:22.399499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:22.413539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe12ed-f25a-4d3f-bb33-cd27e8fa4e0a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:22.421236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d004c-af24-4bfa-86e9-4acaeb8ad7f7 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:22.457584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9ccb5-c938-458c-bebc-dcc0fdaddc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:22.458706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9ccb5-c938-458c-bebc-dcc0fdaddc70 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:22.459283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9ccb5-c938-458c-bebc-dcc0fdaddc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:22.459767 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9ccb5-c938-458c-bebc-dcc0fdaddc70 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:22.460567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9ccb5-c938-458c-bebc-dcc0fdaddc70 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:22.462177 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9ccb5-c938-458c-bebc-dcc0fdaddc70 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:22.466028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a8a90-66da-477f-84a1-eeab6c9394d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:22.466682 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8a90-66da-477f-84a1-eeab6c9394d5 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:22.467053 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8a90-66da-477f-84a1-eeab6c9394d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:22.467330 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8a90-66da-477f-84a1-eeab6c9394d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:22.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-2eb9ccb5-c938-458c-bebc-dcc0fdaddc70 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 14:15:22.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:22.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:22.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:22.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:22.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:22.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:22.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:22.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 14:15:22.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:22.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:22.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:22.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:22.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:22.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:22.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:22.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:22.473796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:22.473796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:22.473796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:22.473796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:22.473796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:22.473796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:22.473796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:22.473796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:22.473796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:22.473796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:22.473796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:22.473796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:22.473796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:22.473796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:22.473796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:22.473796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:22.474866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:22.474866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:22.474866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:22.474866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:22.474866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:22.474866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:22.474866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:22.474866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:22.474866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:22.474866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:22.474866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:22.474866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:22.474866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 14:15:22.474866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:22.475770 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a8a90-66da-477f-84a1-eeab6c9394d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:22.476946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a8a90-66da-477f-84a1-eeab6c9394d5 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:22.477887 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 471/937] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:22.479306 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc3b20-c753-4448-8130-df372f646139 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:22.479750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc3b20-c753-4448-8130-df372f646139 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:22.480032 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc3b20-c753-4448-8130-df372f646139 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:22.481382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 472/938] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:22.483763 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc3b20-c753-4448-8130-df372f646139 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:22.494512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-0e5a8a90-66da-477f-84a1-eeab6c9394d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:22.494512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:22.494512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:22.494512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:22.494512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:22.494512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:22.494512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:22.494512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:22.494512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:22.494512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:22.494512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:22.494512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:22.494512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:15:22.494512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:15:22.494512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:15:22.495512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:15:22.495512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:22.495512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:22.495512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:22.495512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:22.495512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:22.495512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:22.495512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:22.495512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:22.495512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:22.495512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:22.495512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:22.495512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:22.495512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:22.495512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:22.495512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:22.495512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:22.496435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:15:22.496435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:15:22.496435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:15:22.496435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:15:22.496435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:15:22.496435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:15:22.496435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:22.496435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:22.496435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 467/939] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:22.497283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e0141-2717-4fd0-a816-10f04c9903d2 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:22.498027 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e0141-2717-4fd0-a816-10f04c9903d2 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:22.498524 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e0141-2717-4fd0-a816-10f04c9903d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:22.516875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-9a7e0141-2717-4fd0-a816-10f04c9903d2 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:22.518055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9a7e0141-2717-4fd0-a816-10f04c9903d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:22.585729 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 468/940] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:22.587510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e0141-2717-4fd0-a816-10f04c9903d2 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:22.590280 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc3b20-c753-4448-8130-df372f646139 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:22.604140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:22.633866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68011962-2f1f-4300-a900-19c9d1303497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:22.633866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68011962-2f1f-4300-a900-19c9d1303497 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:22.633866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68011962-2f1f-4300-a900-19c9d1303497 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:22.633866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68011962-2f1f-4300-a900-19c9d1303497 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:22.634345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68011962-2f1f-4300-a900-19c9d1303497 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:22.634345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68011962-2f1f-4300-a900-19c9d1303497 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:22.634345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68011962-2f1f-4300-a900-19c9d1303497 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:22.637607 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 473/941] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 5 14:15:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:22.638518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68011962-2f1f-4300-a900-19c9d1303497 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:22.672929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:22.707768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e0141-2717-4fd0-a816-10f04c9903d2 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:22.739975 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68011962-2f1f-4300-a900-19c9d1303497 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:22.751266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23005bcf-c1b6-4bfe-828f-f3e26210b6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:22.752277 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23005bcf-c1b6-4bfe-828f-f3e26210b6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:22.752555 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23005bcf-c1b6-4bfe-828f-f3e26210b6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:22.752825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23005bcf-c1b6-4bfe-828f-f3e26210b6e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:22.753509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23005bcf-c1b6-4bfe-828f-f3e26210b6e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:22.753812 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23005bcf-c1b6-4bfe-828f-f3e26210b6e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:22.755257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23005bcf-c1b6-4bfe-828f-f3e26210b6e9 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:22.772166 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 469/942] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:15:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:22.778103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23005bcf-c1b6-4bfe-828f-f3e26210b6e9 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:22.790938 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e32005-7e74-47be-98ee-eacf00f1992c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:22.791675 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e32005-7e74-47be-98ee-eacf00f1992c None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:22.791957 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e32005-7e74-47be-98ee-eacf00f1992c None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:22.792224 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e32005-7e74-47be-98ee-eacf00f1992c None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:22.792852 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e32005-7e74-47be-98ee-eacf00f1992c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:22.793148 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e32005-7e74-47be-98ee-eacf00f1992c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:22.794930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e32005-7e74-47be-98ee-eacf00f1992c None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:22.806814 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 474/943] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 5 14:15:22 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:22.863112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e32005-7e74-47be-98ee-eacf00f1992c None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:22.900561 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23005bcf-c1b6-4bfe-828f-f3e26210b6e9 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:22.916317 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:22.937672 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:22.942217 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5544b7-47f5-4280-b7a0-0ae0c0ecfc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:22.943249 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5544b7-47f5-4280-b7a0-0ae0c0ecfc3a None admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:22.943546 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5544b7-47f5-4280-b7a0-0ae0c0ecfc3a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:22.943823 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5544b7-47f5-4280-b7a0-0ae0c0ecfc3a None admin] PATH_INFO: `/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/86300433c724483994f59709a3f015db/roles/48a9bca3f43c4943855875bf97ec564f` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:22.946813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5544b7-47f5-4280-b7a0-0ae0c0ecfc3a None admin] RBAC: Authorizing `identity:create_grant(project_id=d9ca42deaed344eea726fe6e32356efc, user_id=86300433c724483994f59709a3f015db, role_id=48a9bca3f43c4943855875bf97ec564f)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:22.950012 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5544b7-47f5-4280-b7a0-0ae0c0ecfc3a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:22.989506 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e32005-7e74-47be-98ee-eacf00f1992c None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:23.025401 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cce7c8-d69c-4c56-ab2c-0c5cf05b85cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:23.026295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cce7c8-d69c-4c56-ab2c-0c5cf05b85cf None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:23.026715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cce7c8-d69c-4c56-ab2c-0c5cf05b85cf None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:23.027163 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cce7c8-d69c-4c56-ab2c-0c5cf05b85cf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:23.027885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cce7c8-d69c-4c56-ab2c-0c5cf05b85cf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:23.029376 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cce7c8-d69c-4c56-ab2c-0c5cf05b85cf None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:23.039329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-f5cce7c8-d69c-4c56-ab2c-0c5cf05b85cf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 14:15:23.039329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:23.039329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:23.039329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:23.039329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:23.039329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:23.039329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:23.039329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:23.039329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 14:15:23.039329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:23.039329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:23.039329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:23.039329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:23.039329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:23.039329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:23.039329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:23.039329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:23.041332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:23.041332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:23.041332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:23.041332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:23.041332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:23.041332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:23.041332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:23.041332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:23.041332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:23.041332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:23.041332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:23.041332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:23.041332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:23.041332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:23.041332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:23.041332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:23.042217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:23.042217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:23.042217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:23.042217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:23.042217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:23.042217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:23.042217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:23.042217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:23.042217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:23.042217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:23.042217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:23.042217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:23.042217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 14:15:23.042217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:23.043927 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 475/945] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 5 14:15:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:23.047059 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 470/945] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:15:22 2022] PUT /identity/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/86300433c724483994f59709a3f015db/roles/48a9bca3f43c4943855875bf97ec564f => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:15:23.049721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cce7c8-d69c-4c56-ab2c-0c5cf05b85cf None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:23.169120 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cce7c8-d69c-4c56-ab2c-0c5cf05b85cf None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:23.177115 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:23.211949 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe6681-d5fb-482a-b768-762fa0195bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:23.212561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6681-d5fb-482a-b768-762fa0195bad None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:23.212837 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6681-d5fb-482a-b768-762fa0195bad None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:23.213102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6681-d5fb-482a-b768-762fa0195bad None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:23.213727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe6681-d5fb-482a-b768-762fa0195bad None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:23.214026 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe6681-d5fb-482a-b768-762fa0195bad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:23.215420 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe6681-d5fb-482a-b768-762fa0195bad None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:23.220482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 476/946] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 5 14:15:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:23.226072 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5544b7-47f5-4280-b7a0-0ae0c0ecfc3a None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:23.298630 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ba3fc-0d4e-42d6-81a1-70f90ec8b5e0 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:23.298976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ba3fc-0d4e-42d6-81a1-70f90ec8b5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:23.299255 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ba3fc-0d4e-42d6-81a1-70f90ec8b5e0 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:23.300609 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 477/947] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:15:23.305746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39b8d5-1096-47bc-814a-472604513243 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:23.306035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39b8d5-1096-47bc-814a-472604513243 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:23.306317 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39b8d5-1096-47bc-814a-472604513243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:23.330819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-fb39b8d5-1096-47bc-814a-472604513243 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:23.332648 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-fb39b8d5-1096-47bc-814a-472604513243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:23.340619 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5544b7-47f5-4280-b7a0-0ae0c0ecfc3a None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:23.383955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aea753-d52f-48b3-8a0d-e826d77c76c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:23.384566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aea753-d52f-48b3-8a0d-e826d77c76c9 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:23.384847 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aea753-d52f-48b3-8a0d-e826d77c76c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:23.385548 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aea753-d52f-48b3-8a0d-e826d77c76c9 None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:23.386181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aea753-d52f-48b3-8a0d-e826d77c76c9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:23.386483 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aea753-d52f-48b3-8a0d-e826d77c76c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:23.388268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aea753-d52f-48b3-8a0d-e826d77c76c9 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:23.394960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 471/948] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 5 14:15:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:23.404096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aea753-d52f-48b3-8a0d-e826d77c76c9 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:23.421180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 478/949] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:23.471886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:23.518009 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aea753-d52f-48b3-8a0d-e826d77c76c9 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:23.553063 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb200d-f3ee-49bb-af27-a65a4b488d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:23.553673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb200d-f3ee-49bb-af27-a65a4b488d04 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:23.553951 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb200d-f3ee-49bb-af27-a65a4b488d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:23.554222 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb200d-f3ee-49bb-af27-a65a4b488d04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:23.554945 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb200d-f3ee-49bb-af27-a65a4b488d04 None admin] RBAC: Adding query filter params (interface=public, service_id=ffb5ded7bdf14bfd8cea4051e9eddcb8, region_id=RegionOne) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:23.555790 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb200d-f3ee-49bb-af27-a65a4b488d04 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:23.557161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb200d-f3ee-49bb-af27-a65a4b488d04 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:23.562697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 472/950] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Sep 5 14:15:23 2022] GET /identity/v3/endpoints?service_id=ffb5ded7bdf14bfd8cea4051e9eddcb8&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:23.571950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39b8d5-1096-47bc-814a-472604513243 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:23.674430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:23.678030 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39b8d5-1096-47bc-814a-472604513243 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:23.691127 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb200d-f3ee-49bb-af27-a65a4b488d04 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:23.715846 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a46fa-8695-47e6-95dd-4df1d27a5ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:23.716840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a46fa-8695-47e6-95dd-4df1d27a5ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:23.717408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a46fa-8695-47e6-95dd-4df1d27a5ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:23.717894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a46fa-8695-47e6-95dd-4df1d27a5ae9 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:23.718705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a46fa-8695-47e6-95dd-4df1d27a5ae9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:23.720468 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a46fa-8695-47e6-95dd-4df1d27a5ae9 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:23.727816 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 479/951] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 5 14:15:23 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:15:23.806292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb200d-f3ee-49bb-af27-a65a4b488d04 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:23.828220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:23.843497 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac780db-b1b1-4993-b249-4ca2de55c82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:23.844113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac780db-b1b1-4993-b249-4ca2de55c82b None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:23.844389 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac780db-b1b1-4993-b249-4ca2de55c82b None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:23.844696 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac780db-b1b1-4993-b249-4ca2de55c82b None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:23.845357 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac780db-b1b1-4993-b249-4ca2de55c82b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:23.846834 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac780db-b1b1-4993-b249-4ca2de55c82b None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:23.849539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: WARNING keystone.server.flask.common [None req-aac780db-b1b1-4993-b249-4ca2de55c82b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 05 14:15:23.913022 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.sql.core [None req-aac780db-b1b1-4993-b249-4ca2de55c82b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 05 14:15:23.913022 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 05 14:15:23.913022 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [parameters: {'id': 'ba0d6da9bb9d45aabbd1210fe0b5a6e9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 05 14:15:23.913022 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81594) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 05 14:15:23.930316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-aac780db-b1b1-4993-b249-4ca2de55c82b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ba0d6da9bb9d45aabbd1210fe0b5a6e9.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ba0d6da9bb9d45aabbd1210fe0b5a6e9. Sep 05 14:15:23.930316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:23.930316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 14:15:23.930316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 14:15:23.930316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 14:15:23.930316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 14:15:23.930316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 14:15:23.930316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 14:15:23.930316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 14:15:23.930316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application conn.query(q) Sep 05 14:15:23.930316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 14:15:23.930316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 14:15:23.930316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application result.read() Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 05 14:15:23.931260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 05 14:15:23.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 14:15:23.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application next(self.gen) Sep 05 14:15:23.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 05 14:15:23.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application yield resource Sep 05 14:15:23.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 14:15:23.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application next(self.gen) Sep 05 14:15:23.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 05 14:15:23.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self.session.rollback() Sep 05 14:15:23.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 05 14:15:23.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self.force_reraise() Sep 05 14:15:23.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 05 14:15:23.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise self.value Sep 05 14:15:23.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 05 14:15:23.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 05 14:15:23.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 05 14:15:23.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application session.commit() Sep 05 14:15:23.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 05 14:15:23.933458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 05 14:15:23.933458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 05 14:15:23.933458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._prepare_impl() Sep 05 14:15:23.933458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 05 14:15:23.933458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self.session.flush() Sep 05 14:15:23.933458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 05 14:15:23.933458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._flush(objects) Sep 05 14:15:23.933458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 05 14:15:23.933458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 05 14:15:23.933458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 05 14:15:23.933458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application compat.raise_( Sep 05 14:15:23.933458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 14:15:23.933458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception Sep 05 14:15:23.933458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 05 14:15:23.933458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application flush_context.execute() Sep 05 14:15:23.933458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 05 14:15:23.933458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rec.execute(self) Sep 05 14:15:23.940839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 05 14:15:23.940839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 05 14:15:23.940839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 05 14:15:23.940839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 05 14:15:23.940839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 05 14:15:23.940839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 05 14:15:23.940839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 05 14:15:23.940839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 05 14:15:23.940839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 05 14:15:23.940839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 05 14:15:23.940839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 05 14:15:23.940839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 05 14:15:23.940839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 05 14:15:23.940839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 05 14:15:23.940839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 05 14:15:23.943153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 05 14:15:23.943153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 14:15:23.943153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception Sep 05 14:15:23.943153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 14:15:23.943153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 14:15:23.943153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 14:15:23.943153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 14:15:23.943153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 14:15:23.943153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 14:15:23.943153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 14:15:23.943153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application conn.query(q) Sep 05 14:15:23.943153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 14:15:23.943153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 14:15:23.943153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 14:15:23.943153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application result.read() Sep 05 14:15:23.943153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 14:15:23.943153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 14:15:23.945335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 14:15:23.945335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 14:15:23.945335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 14:15:23.945335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 14:15:23.945335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 14:15:23.945335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 14:15:23.945335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 05 14:15:23.945335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 05 14:15:23.945335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [parameters: {'id': 'ba0d6da9bb9d45aabbd1210fe0b5a6e9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 05 14:15:23.945335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 05 14:15:23.945335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:23.945335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:23.945335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:23.945335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:23.945335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 05 14:15:23.947843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 05 14:15:23.947843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 05 14:15:23.947843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 05 14:15:23.947843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 05 14:15:23.947843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:23.947843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:23.947843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:23.947843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:23.947843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:23.947843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:23.947843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:23.947843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:23.947843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:23.947843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:23.947843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:23.947843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:23.947843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:23.950147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:23.950147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 05 14:15:23.950147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 05 14:15:23.950147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:23.950147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:23.950147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 05 14:15:23.950147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 05 14:15:23.950147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ba0d6da9bb9d45aabbd1210fe0b5a6e9. Sep 05 14:15:23.950147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:23.950147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 473/952] 10.222.0.15 () {60 vars in 1140 bytes} [Mon Sep 5 14:15:23 2022] POST /identity/v3/projects => generated 262 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:23.955891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a46fa-8695-47e6-95dd-4df1d27a5ae9 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:24.049442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a46fa-8695-47e6-95dd-4df1d27a5ae9 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:24.086907 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb50fd-ec88-40be-8811-99a15ac4e269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:24.087552 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb50fd-ec88-40be-8811-99a15ac4e269 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:24.087836 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb50fd-ec88-40be-8811-99a15ac4e269 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:24.088113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb50fd-ec88-40be-8811-99a15ac4e269 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:24.095209 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb50fd-ec88-40be-8811-99a15ac4e269 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:24.096679 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb50fd-ec88-40be-8811-99a15ac4e269 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:24.108851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-fcdb50fd-ec88-40be-8811-99a15ac4e269 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:24.108851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:24.108851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:24.108851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:24.108851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:24.108851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:24.108851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:24.108851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:24.108851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:24.108851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:24.108851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:24.108851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:24.108851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:15:24.108851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:15:24.108851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:15:24.110997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:15:24.110997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:24.110997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:24.110997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:24.110997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:24.110997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:24.110997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:24.110997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:24.110997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:24.110997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:24.110997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:24.110997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:24.110997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:24.110997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:24.110997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:24.110997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:24.110997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:24.113192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:15:24.113192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:15:24.113192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:15:24.113192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:15:24.113192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:15:24.113192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:15:24.113192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:24.113192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:24.118810 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 480/953] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:24.120139 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac780db-b1b1-4993-b249-4ca2de55c82b None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:24.232669 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac780db-b1b1-4993-b249-4ca2de55c82b None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:24.267075 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5297f-1225-46fa-8a20-868015e5b5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:24.267694 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5297f-1225-46fa-8a20-868015e5b5de None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:24.267980 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5297f-1225-46fa-8a20-868015e5b5de None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:24.268322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5297f-1225-46fa-8a20-868015e5b5de None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:24.268989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5297f-1225-46fa-8a20-868015e5b5de None admin] RBAC: Adding query filter params (name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:24.269349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5297f-1225-46fa-8a20-868015e5b5de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:24.271096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5297f-1225-46fa-8a20-868015e5b5de None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:24.292102 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 474/954] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 5 14:15:24 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:24.863172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ad995-f3c6-46ea-8041-41120375c353 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:24.863657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ad995-f3c6-46ea-8041-41120375c353 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:24.864040 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ad995-f3c6-46ea-8041-41120375c353 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:24.865692 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 481/955] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:24.872675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8dfad-e8a4-4a25-9306-ad55ca4e8b9c None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:24.872675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8dfad-e8a4-4a25-9306-ad55ca4e8b9c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:24.872675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8dfad-e8a4-4a25-9306-ad55ca4e8b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:24.891722 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-bee8dfad-e8a4-4a25-9306-ad55ca4e8b9c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:24.893279 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-bee8dfad-e8a4-4a25-9306-ad55ca4e8b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:24.925223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:24.964768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 475/956] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:25.030652 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d41b5-74bb-4d80-924d-cfe2f86dfcff None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:25.031232 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d41b5-74bb-4d80-924d-cfe2f86dfcff None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:25.031232 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d41b5-74bb-4d80-924d-cfe2f86dfcff None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:25.032786 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 482/957] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:25.037962 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20425de2-1ec7-4861-884c-fe6e8b73e112 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:25.038428 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20425de2-1ec7-4861-884c-fe6e8b73e112 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:25.038902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20425de2-1ec7-4861-884c-fe6e8b73e112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:25.061614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-20425de2-1ec7-4861-884c-fe6e8b73e112 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:25.062800 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-20425de2-1ec7-4861-884c-fe6e8b73e112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:25.132039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 476/958] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:25.248355 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d41b5-74bb-4d80-924d-cfe2f86dfcff None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:25.275619 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:25.354373 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d41b5-74bb-4d80-924d-cfe2f86dfcff None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:25.390153 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ddc2b-4cdd-45b3-9a74-5d8ac3772469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:25.390676 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ddc2b-4cdd-45b3-9a74-5d8ac3772469 None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:25.390988 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ddc2b-4cdd-45b3-9a74-5d8ac3772469 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:25.391261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ddc2b-4cdd-45b3-9a74-5d8ac3772469 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:25.391971 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ddc2b-4cdd-45b3-9a74-5d8ac3772469 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:25.393341 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ddc2b-4cdd-45b3-9a74-5d8ac3772469 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:25.398641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20425de2-1ec7-4861-884c-fe6e8b73e112 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:25.491250 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 483/959] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 5 14:15:25 2022] POST /identity/v3/services => generated 242 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:15:25.501581 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:25.507974 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20425de2-1ec7-4861-884c-fe6e8b73e112 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:25.555827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d574c26-fb61-4f35-92d2-2bed4593e2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:25.556538 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d574c26-fb61-4f35-92d2-2bed4593e2bf None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:25.556826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d574c26-fb61-4f35-92d2-2bed4593e2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:25.557151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d574c26-fb61-4f35-92d2-2bed4593e2bf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:25.569325 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d574c26-fb61-4f35-92d2-2bed4593e2bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:15:25.574622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-8d574c26-fb61-4f35-92d2-2bed4593e2bf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 14:15:25.574622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:25.574622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:25.574622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:25.574622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:25.574622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:25.574622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:25.574622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:25.574622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:25.574622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:25.574622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:25.574622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:25.574622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:15:25.574622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:15:25.574622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:15:25.574622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:15:25.575692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:15:25.575692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:15:25.575692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:15:25.575692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:15:25.575692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:25.575692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:25.575692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:25.575692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:25.575692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:25.575692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:25.575692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:25.575692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:25.575692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:25.575692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:25.575692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:25.575692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:25.576586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:25.576586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:25.576586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:25.576586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:25.576586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:15:25.576586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:15:25.576586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:15:25.576586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:15:25.576586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:15:25.576586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:15:25.576586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 14:15:25.576586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:25.576586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 477/960] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Sep 5 14:15:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:25.583984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ddc2b-4cdd-45b3-9a74-5d8ac3772469 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:25.676862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:25.693143 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ddc2b-4cdd-45b3-9a74-5d8ac3772469 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:25.729164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af9a36-0a0e-4f65-9d99-4d066789f49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:25.729829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af9a36-0a0e-4f65-9d99-4d066789f49d None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:25.730111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af9a36-0a0e-4f65-9d99-4d066789f49d None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:25.730382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af9a36-0a0e-4f65-9d99-4d066789f49d None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:25.731122 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af9a36-0a0e-4f65-9d99-4d066789f49d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:25.731429 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af9a36-0a0e-4f65-9d99-4d066789f49d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:25.732796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af9a36-0a0e-4f65-9d99-4d066789f49d None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:25.741959 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 484/961] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Sep 5 14:15:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:25.747570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d574c26-fb61-4f35-92d2-2bed4593e2bf None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:25.791004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb78aa-f53c-4885-a916-9474a3256f64 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:25.791353 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb78aa-f53c-4885-a916-9474a3256f64 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:25.791595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb78aa-f53c-4885-a916-9474a3256f64 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:25.793187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 485/962] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:15:25.798169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743c36a-2c24-41fc-ad00-2be8d9ce149f None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:25.798454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743c36a-2c24-41fc-ad00-2be8d9ce149f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:25.798734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743c36a-2c24-41fc-ad00-2be8d9ce149f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:25.818079 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-6743c36a-2c24-41fc-ad00-2be8d9ce149f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:25.819361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-6743c36a-2c24-41fc-ad00-2be8d9ce149f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:25.865869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d574c26-fb61-4f35-92d2-2bed4593e2bf None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:25.895544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 486/963] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:25.903406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b8498-c844-46e2-9d74-25eac159df3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:25.904427 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b8498-c844-46e2-9d74-25eac159df3d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:25.904954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b8498-c844-46e2-9d74-25eac159df3d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:25.905462 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b8498-c844-46e2-9d74-25eac159df3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:25.917026 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b8498-c844-46e2-9d74-25eac159df3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:25.918779 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b8498-c844-46e2-9d74-25eac159df3d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:25.934920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-590b8498-c844-46e2-9d74-25eac159df3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:25.934920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:25.934920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:25.934920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:25.934920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:25.934920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:25.934920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:25.934920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:25.934920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:25.934920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:25.934920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:25.934920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:25.934920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:25.934920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:25.934920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:25.934920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:25.936294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:25.936294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:25.936294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:25.936294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:25.936294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:25.936294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:25.936294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:25.936294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:25.936294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:25.936294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:25.936294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:25.936294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:25.936294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:25.936294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:25.936294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:25.936294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:25.936294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:25.937236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:25.937236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:25.937236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:25.939309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 478/964] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:25.950832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743c36a-2c24-41fc-ad00-2be8d9ce149f None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:26.025030 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:26.048879 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743c36a-2c24-41fc-ad00-2be8d9ce149f None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:26.084933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f75b5-27f7-4218-acf9-e6ad03a6b8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:26.085543 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f75b5-27f7-4218-acf9-e6ad03a6b8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:26.085822 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f75b5-27f7-4218-acf9-e6ad03a6b8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:26.086097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f75b5-27f7-4218-acf9-e6ad03a6b8d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:26.087303 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f75b5-27f7-4218-acf9-e6ad03a6b8d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:26.087605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f75b5-27f7-4218-acf9-e6ad03a6b8d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:26.089041 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f75b5-27f7-4218-acf9-e6ad03a6b8d2 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:26.101241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 487/965] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:26.106923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b8498-c844-46e2-9d74-25eac159df3d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:26.110986 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:26.202032 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f75b5-27f7-4218-acf9-e6ad03a6b8d2 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:26.220535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b8498-c844-46e2-9d74-25eac159df3d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:26.259321 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf266b54-07fb-413f-9da3-5ba99eed8076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:26.260185 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf266b54-07fb-413f-9da3-5ba99eed8076 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:26.260652 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf266b54-07fb-413f-9da3-5ba99eed8076 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:26.261091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf266b54-07fb-413f-9da3-5ba99eed8076 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:26.269444 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf266b54-07fb-413f-9da3-5ba99eed8076 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:26.271166 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf266b54-07fb-413f-9da3-5ba99eed8076 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:26.283590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-bf266b54-07fb-413f-9da3-5ba99eed8076 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 14:15:26.283590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:26.283590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:26.283590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:26.283590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:26.283590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:26.283590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:26.283590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:26.283590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:26.283590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:26.283590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:26.283590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:26.283590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:15:26.283590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:15:26.283590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:15:26.283590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:26.284783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:26.285990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:26.285990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:26.285990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:15:26.285990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:15:26.285990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:15:26.285990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:15:26.285990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:15:26.285990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:15:26.285990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 14:15:26.285990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:26.292880 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 479/966] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 5 14:15:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:26.293806 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf266b54-07fb-413f-9da3-5ba99eed8076 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:26.324799 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f75b5-27f7-4218-acf9-e6ad03a6b8d2 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:26.360473 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fe6b7-be67-4c81-ab1e-e82111208658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:26.361108 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fe6b7-be67-4c81-ab1e-e82111208658 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:26.361391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fe6b7-be67-4c81-ab1e-e82111208658 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:26.361667 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fe6b7-be67-4c81-ab1e-e82111208658 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:26.362249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98fe6b7-be67-4c81-ab1e-e82111208658 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:26.363645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98fe6b7-be67-4c81-ab1e-e82111208658 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:26.365736 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:26.366828 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:26.373792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-f98fe6b7-be67-4c81-ab1e-e82111208658 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 14:15:26.373792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:26.373792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:26.373792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:26.373792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:26.373792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:26.373792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:26.373792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:26.373792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 14:15:26.373792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:26.373792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:26.373792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:26.373792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:26.373792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:26.373792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:26.373792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:26.373792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:26.383583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:26.383583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:26.383583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:26.383583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:26.383583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:26.383583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:26.383583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:26.383583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:26.383583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:26.383583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:26.383583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:26.383583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:26.383583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:26.383583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:26.383583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:26.383583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:26.384618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:26.384618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:26.384618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:26.384618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:26.384618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:26.384618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:26.384618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:26.384618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:26.384618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:26.384618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:26.384618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:26.384618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:26.384618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 14:15:26.384618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:26.384618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 488/967] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 5 14:15:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:26.385466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ebfee-4a11-4ac2-aa7f-2e9f2369cbc7 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:26.385466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ebfee-4a11-4ac2-aa7f-2e9f2369cbc7 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:26.385466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ebfee-4a11-4ac2-aa7f-2e9f2369cbc7 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:26.385466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 489/968] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:26.385466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ebfee-4a11-4ac2-aa7f-2e9f2369cbc7 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:26.410698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf266b54-07fb-413f-9da3-5ba99eed8076 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:26.445686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1834b89-28e1-485a-9bb3-173a4c016704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:26.446318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1834b89-28e1-485a-9bb3-173a4c016704 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:26.446601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1834b89-28e1-485a-9bb3-173a4c016704 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:26.446906 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1834b89-28e1-485a-9bb3-173a4c016704 None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:26.447636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1834b89-28e1-485a-9bb3-173a4c016704 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:26.447967 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1834b89-28e1-485a-9bb3-173a4c016704 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:26.449412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1834b89-28e1-485a-9bb3-173a4c016704 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:26.471821 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 480/969] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Sep 5 14:15:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:26.471821 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8f713-81de-43e3-a0bd-083e67ae73e1 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:26.471821 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8f713-81de-43e3-a0bd-083e67ae73e1 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:26.471821 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8f713-81de-43e3-a0bd-083e67ae73e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:26.487444 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-45f8f713-81de-43e3-a0bd-083e67ae73e1 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:26.488620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-45f8f713-81de-43e3-a0bd-083e67ae73e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:26.506536 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ebfee-4a11-4ac2-aa7f-2e9f2369cbc7 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:26.542896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857cda4-a3cd-4d22-91aa-1b57a9d3f035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:26.543530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857cda4-a3cd-4d22-91aa-1b57a9d3f035 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:26.543840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857cda4-a3cd-4d22-91aa-1b57a9d3f035 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:26.544194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857cda4-a3cd-4d22-91aa-1b57a9d3f035 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:26.544815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857cda4-a3cd-4d22-91aa-1b57a9d3f035 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:26.545116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857cda4-a3cd-4d22-91aa-1b57a9d3f035 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:26.546583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857cda4-a3cd-4d22-91aa-1b57a9d3f035 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:26.555604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 490/970] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 5 14:15:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:26.556735 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857cda4-a3cd-4d22-91aa-1b57a9d3f035 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:26.557263 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 481/971] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:26.558348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8f713-81de-43e3-a0bd-083e67ae73e1 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:26.621855 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:26.634574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:26.674219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857cda4-a3cd-4d22-91aa-1b57a9d3f035 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:26.687469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8f713-81de-43e3-a0bd-083e67ae73e1 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:26.722213 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b2d63-cef2-46b2-ad75-006339a5b101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:26.723124 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b2d63-cef2-46b2-ad75-006339a5b101 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:26.723593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b2d63-cef2-46b2-ad75-006339a5b101 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:26.724048 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b2d63-cef2-46b2-ad75-006339a5b101 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:26.724574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2a0f6-55a8-4249-ac09-933540cde0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:26.725187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2a0f6-55a8-4249-ac09-933540cde0bc None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:26.725470 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2a0f6-55a8-4249-ac09-933540cde0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:26.725747 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2a0f6-55a8-4249-ac09-933540cde0bc None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:26.726373 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2a0f6-55a8-4249-ac09-933540cde0bc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:26.726676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2a0f6-55a8-4249-ac09-933540cde0bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:26.728639 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2a0f6-55a8-4249-ac09-933540cde0bc None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:26.733945 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 482/972] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 5 14:15:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:26.739387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2a0f6-55a8-4249-ac09-933540cde0bc None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:26.807082 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b2d63-cef2-46b2-ad75-006339a5b101 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:26.808561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b2d63-cef2-46b2-ad75-006339a5b101 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:26.845719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2a0f6-55a8-4249-ac09-933540cde0bc None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:26.863915 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:26.877804 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:26.882444 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd8bd4-c2e9-4f74-aeba-5743b8fcbbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:26.882658 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd8bd4-c2e9-4f74-aeba-5743b8fcbbe6 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:26.882989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd8bd4-c2e9-4f74-aeba-5743b8fcbbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:26.883820 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd8bd4-c2e9-4f74-aeba-5743b8fcbbe6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:26.884477 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd8bd4-c2e9-4f74-aeba-5743b8fcbbe6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:26.885844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd8bd4-c2e9-4f74-aeba-5743b8fcbbe6 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:26.916634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 491/973] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:15:26 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:15:26.917578 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b2d63-cef2-46b2-ad75-006339a5b101 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:26.987223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 483/974] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:15:26 2022] POST /identity/v3/endpoints => generated 349 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:15:26.987972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd8bd4-c2e9-4f74-aeba-5743b8fcbbe6 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:27.020193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b2d63-cef2-46b2-ad75-006339a5b101 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:27.067716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dd880-8004-4e34-9493-e0c5f8409c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:27.069034 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dd880-8004-4e34-9493-e0c5f8409c24 None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:27.069667 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dd880-8004-4e34-9493-e0c5f8409c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:27.070118 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dd880-8004-4e34-9493-e0c5f8409c24 None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:27.071560 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1dd880-8004-4e34-9493-e0c5f8409c24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:27.073289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1dd880-8004-4e34-9493-e0c5f8409c24 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:27.078945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: WARNING keystone.server.flask.common [None req-4f1dd880-8004-4e34-9493-e0c5f8409c24 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 05 14:15:27.124957 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.sql.core [None req-4f1dd880-8004-4e34-9493-e0c5f8409c24 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 05 14:15:27.124957 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 05 14:15:27.124957 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [parameters: {'user_id': '8468171755e24fe5a8f8eaf5e07f2e76', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 05 14:15:27.124957 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81595) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 05 14:15:27.131158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd8bd4-c2e9-4f74-aeba-5743b8fcbbe6 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:27.136268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-4f1dd880-8004-4e34-9493-e0c5f8409c24 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 05 14:15:27.136268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:27.136268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 14:15:27.136268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 14:15:27.136268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 14:15:27.136268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 14:15:27.136268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 14:15:27.136268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 14:15:27.136268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 14:15:27.136268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application conn.query(q) Sep 05 14:15:27.136268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 14:15:27.136268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 14:15:27.136268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 14:15:27.136268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application result.read() Sep 05 14:15:27.136268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 14:15:27.137242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application next(self.gen) Sep 05 14:15:27.138233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 05 14:15:27.138233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application yield resource Sep 05 14:15:27.138233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 14:15:27.138233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application next(self.gen) Sep 05 14:15:27.138233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 05 14:15:27.138233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self.session.rollback() Sep 05 14:15:27.138233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 05 14:15:27.138233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self.force_reraise() Sep 05 14:15:27.138233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 05 14:15:27.138233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise self.value Sep 05 14:15:27.138233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 05 14:15:27.138233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 05 14:15:27.138233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 05 14:15:27.138233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application session.commit() Sep 05 14:15:27.138233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 05 14:15:27.138233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 05 14:15:27.138233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 05 14:15:27.141263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._prepare_impl() Sep 05 14:15:27.141263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 05 14:15:27.141263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self.session.flush() Sep 05 14:15:27.141263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 05 14:15:27.141263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._flush(objects) Sep 05 14:15:27.141263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 05 14:15:27.141263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 05 14:15:27.141263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 05 14:15:27.141263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application compat.raise_( Sep 05 14:15:27.141263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 14:15:27.141263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception Sep 05 14:15:27.141263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 05 14:15:27.141263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application flush_context.execute() Sep 05 14:15:27.141263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 05 14:15:27.141263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rec.execute(self) Sep 05 14:15:27.141263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 05 14:15:27.141263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 05 14:15:27.142205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 05 14:15:27.142205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 05 14:15:27.142205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 05 14:15:27.142205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 05 14:15:27.142205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 05 14:15:27.142205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 05 14:15:27.142205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 05 14:15:27.142205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 05 14:15:27.142205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 05 14:15:27.142205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 05 14:15:27.142205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 05 14:15:27.142205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 05 14:15:27.142205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 05 14:15:27.142205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 05 14:15:27.142205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 14:15:27.142205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception Sep 05 14:15:27.144164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 14:15:27.144164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 14:15:27.144164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 14:15:27.144164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 14:15:27.144164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 14:15:27.144164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 14:15:27.144164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 14:15:27.144164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application conn.query(q) Sep 05 14:15:27.144164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 14:15:27.144164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 14:15:27.144164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 14:15:27.144164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application result.read() Sep 05 14:15:27.144164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 14:15:27.144164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 14:15:27.144164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 14:15:27.144164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 14:15:27.146337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 14:15:27.146337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 14:15:27.146337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 14:15:27.146337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 14:15:27.146337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 05 14:15:27.146337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 05 14:15:27.146337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [parameters: {'user_id': '8468171755e24fe5a8f8eaf5e07f2e76', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 05 14:15:27.146337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 05 14:15:27.146337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:27.146337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:27.146337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:27.146337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:27.146337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:27.146337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:27.146337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:27.148331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:27.148331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:27.148331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:27.148331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:27.148331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:27.148331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:27.148331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:27.148331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 05 14:15:27.148331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 05 14:15:27.148331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:27.148331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:27.148331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:15:27.148331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:27.148331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:15:27.148331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:27.148331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 05 14:15:27.148331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 05 14:15:27.151350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 05 14:15:27.151350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 05 14:15:27.151350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 05 14:15:27.151350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 05 14:15:27.151350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 05 14:15:27.151350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:27.151350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 492/975] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Sep 5 14:15:26 2022] POST /identity/v3/users => generated 160 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:27.151350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dd880-8004-4e34-9493-e0c5f8409c24 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:27.176651 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:27.191842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb9e2e-5b00-4a4a-a82c-22b4e4a12805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:27.194405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb9e2e-5b00-4a4a-a82c-22b4e4a12805 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:27.194405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb9e2e-5b00-4a4a-a82c-22b4e4a12805 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:27.194405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb9e2e-5b00-4a4a-a82c-22b4e4a12805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:27.210398 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb9e2e-5b00-4a4a-a82c-22b4e4a12805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:27.212491 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb9e2e-5b00-4a4a-a82c-22b4e4a12805 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:27.228225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-aecb9e2e-5b00-4a4a-a82c-22b4e4a12805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:27.228225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:27.228225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:27.228225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:27.228225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:27.228225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:27.228225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:27.228225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:27.228225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:27.228225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:27.228225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:27.228225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:27.228225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:27.228225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:27.228225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:27.228225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:27.229251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:27.229251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:27.229251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:27.229251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:27.229251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:27.229251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:27.229251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:27.229251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:27.229251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:27.229251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:27.229251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:27.229251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:27.229251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:27.229251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:27.229251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:27.229251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:27.229251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:27.230184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:27.230184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:27.230184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:27.230184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 484/976] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:27.235060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb9e2e-5b00-4a4a-a82c-22b4e4a12805 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:27.267953 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:27.281237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dd880-8004-4e34-9493-e0c5f8409c24 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:27.319049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1fc54-4dad-4c7b-8774-1e1edc67df5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:27.320025 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1fc54-4dad-4c7b-8774-1e1edc67df5b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:27.320371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1fc54-4dad-4c7b-8774-1e1edc67df5b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:27.320732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1fc54-4dad-4c7b-8774-1e1edc67df5b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:27.330289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1fc54-4dad-4c7b-8774-1e1edc67df5b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:27.332060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1fc54-4dad-4c7b-8774-1e1edc67df5b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:27.347319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-60e1fc54-4dad-4c7b-8774-1e1edc67df5b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 14:15:27.347319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:27.347319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:27.347319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:27.347319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:27.347319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:27.347319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:27.347319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:27.347319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:27.347319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:27.347319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:27.347319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:27.347319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:15:27.347319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:15:27.347319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:15:27.347319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:27.348335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:27.349321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:27.349321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:27.349321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:15:27.349321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:15:27.349321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:15:27.349321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:15:27.349321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:15:27.349321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:15:27.349321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 14:15:27.349321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:27.349321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 493/977] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:15:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:27.355779 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1fc54-4dad-4c7b-8774-1e1edc67df5b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:27.367897 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb9e2e-5b00-4a4a-a82c-22b4e4a12805 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:27.412824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1876f16-42da-46a9-9b8d-85078598a4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:27.414005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1876f16-42da-46a9-9b8d-85078598a4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:27.414534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1876f16-42da-46a9-9b8d-85078598a4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:27.415195 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1876f16-42da-46a9-9b8d-85078598a4a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:27.416420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1876f16-42da-46a9-9b8d-85078598a4a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:27.416921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1876f16-42da-46a9-9b8d-85078598a4a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:27.418482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1876f16-42da-46a9-9b8d-85078598a4a3 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:27.435073 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 485/978] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:27.442497 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1876f16-42da-46a9-9b8d-85078598a4a3 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:27.461082 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:27.480958 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1fc54-4dad-4c7b-8774-1e1edc67df5b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:27.522890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424d283-2a9f-4f86-b9cc-eda859f53b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:27.523878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424d283-2a9f-4f86-b9cc-eda859f53b18 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:27.524378 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424d283-2a9f-4f86-b9cc-eda859f53b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:27.524897 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424d283-2a9f-4f86-b9cc-eda859f53b18 None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:27.526004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424d283-2a9f-4f86-b9cc-eda859f53b18 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:27.526472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424d283-2a9f-4f86-b9cc-eda859f53b18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:27.528017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424d283-2a9f-4f86-b9cc-eda859f53b18 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:27.542958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:27.558550 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1876f16-42da-46a9-9b8d-85078598a4a3 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:27.570494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 494/979] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 5 14:15:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:27.572726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d5209-2143-40b0-baec-b8a28160e82f None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:27.573014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d5209-2143-40b0-baec-b8a28160e82f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:27.573287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d5209-2143-40b0-baec-b8a28160e82f None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:27.576946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 495/980] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:27.585607 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e64217-8c98-4047-aa03-deb7e31b6305 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:27.585967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e64217-8c98-4047-aa03-deb7e31b6305 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:27.586155 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e64217-8c98-4047-aa03-deb7e31b6305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:27.607660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa352d-3238-44e3-8f1f-13c8763b09a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:27.608704 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa352d-3238-44e3-8f1f-13c8763b09a9 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:27.609188 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa352d-3238-44e3-8f1f-13c8763b09a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:27.609679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa352d-3238-44e3-8f1f-13c8763b09a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:27.610014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-f2e64217-8c98-4047-aa03-deb7e31b6305 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:27.611841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f2e64217-8c98-4047-aa03-deb7e31b6305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:27.619137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa352d-3238-44e3-8f1f-13c8763b09a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:27.620954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa352d-3238-44e3-8f1f-13c8763b09a9 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:27.633701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-a4fa352d-3238-44e3-8f1f-13c8763b09a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:27.633701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:27.633701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:27.633701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:27.633701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:27.633701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:27.633701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:27.633701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:27.633701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:27.633701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:27.633701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:27.633701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:27.633701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:15:27.633701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:15:27.633701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:15:27.634676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:15:27.634676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:27.634676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:27.634676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:27.634676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:27.634676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:27.634676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:27.634676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:27.634676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:27.634676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:27.634676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:27.634676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:27.634676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:27.634676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:27.634676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:27.634676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:27.634676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:27.635629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:15:27.635629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:15:27.635629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:15:27.635629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:15:27.635629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:15:27.635629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:15:27.635629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:27.635629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:27.637837 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 486/981] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:27.643875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa352d-3238-44e3-8f1f-13c8763b09a9 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:27.710936 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 496/982] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:27.767199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa352d-3238-44e3-8f1f-13c8763b09a9 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:27.805816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20959cb5-aad3-437e-8388-7ff228593656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:27.807479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20959cb5-aad3-437e-8388-7ff228593656 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:27.808401 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20959cb5-aad3-437e-8388-7ff228593656 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:27.809180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20959cb5-aad3-437e-8388-7ff228593656 None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:27.810483 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20959cb5-aad3-437e-8388-7ff228593656 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:27.811012 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20959cb5-aad3-437e-8388-7ff228593656 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:27.812664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20959cb5-aad3-437e-8388-7ff228593656 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:27.826563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 487/983] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:15:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:27.832875 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e64217-8c98-4047-aa03-deb7e31b6305 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:27.910978 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:27.943365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e64217-8c98-4047-aa03-deb7e31b6305 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:27.976406 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8476b8-500a-4b4a-9a77-482b76e5eac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:27.977549 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8476b8-500a-4b4a-9a77-482b76e5eac7 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:27.977829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8476b8-500a-4b4a-9a77-482b76e5eac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:27.978096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8476b8-500a-4b4a-9a77-482b76e5eac7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:27.978822 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8476b8-500a-4b4a-9a77-482b76e5eac7 None admin] RBAC: Adding query filter params (role.id=48a9bca3f43c4943855875bf97ec564f, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=86300433c724483994f59709a3f015db) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:27.979133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8476b8-500a-4b4a-9a77-482b76e5eac7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:27.980615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8476b8-500a-4b4a-9a77-482b76e5eac7 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:27.983139 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20959cb5-aad3-437e-8388-7ff228593656 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:27.987081 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 497/984] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:15:27 2022] GET /identity/v3/role_assignments?user.id=86300433c724483994f59709a3f015db&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=48a9bca3f43c4943855875bf97ec564f => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:27.987211 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:28.105182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20959cb5-aad3-437e-8388-7ff228593656 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:28.142110 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ef0ba-4368-48e2-ac08-8a48e338f56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:28.143145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ef0ba-4368-48e2-ac08-8a48e338f56e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:28.143426 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ef0ba-4368-48e2-ac08-8a48e338f56e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:28.143701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ef0ba-4368-48e2-ac08-8a48e338f56e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:28.144285 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ef0ba-4368-48e2-ac08-8a48e338f56e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:28.145667 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ef0ba-4368-48e2-ac08-8a48e338f56e None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:28.155125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-064ef0ba-4368-48e2-ac08-8a48e338f56e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 14:15:28.155125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:28.155125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:28.155125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:28.155125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:28.155125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:28.155125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:28.155125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:28.155125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 14:15:28.155125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:28.155125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:28.155125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:28.155125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:28.155125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:28.155125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:28.155125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:28.155125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:28.156128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:28.156128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:28.156128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:28.156128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:28.156128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:28.156128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:28.156128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:28.156128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:28.156128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:28.156128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:28.156128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:28.156128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:28.156128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:28.156128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:28.156128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:28.156128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:28.156966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:28.156966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:28.156966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:28.156966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:28.156966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:28.156966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:28.156966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:28.156966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:28.156966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:28.156966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:28.156966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:28.156966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:28.156966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 14:15:28.156966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:28.156966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 488/985] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:28.162368 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8476b8-500a-4b4a-9a77-482b76e5eac7 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:28.268698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8476b8-500a-4b4a-9a77-482b76e5eac7 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:28.302680 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f246b4a-59a0-42f8-9fba-bd74a80dd309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:28.303338 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f246b4a-59a0-42f8-9fba-bd74a80dd309 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:28.303619 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f246b4a-59a0-42f8-9fba-bd74a80dd309 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:28.303968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f246b4a-59a0-42f8-9fba-bd74a80dd309 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:28.304593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f246b4a-59a0-42f8-9fba-bd74a80dd309 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:28.304951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f246b4a-59a0-42f8-9fba-bd74a80dd309 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:28.306457 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f246b4a-59a0-42f8-9fba-bd74a80dd309 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:28.307604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:28.312503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 498/986] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 5 14:15:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:28.318469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ef0ba-4368-48e2-ac08-8a48e338f56e None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:28.385853 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:28.419269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ef0ba-4368-48e2-ac08-8a48e338f56e None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:28.454004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934bea4-c0df-4d62-99c1-cd1492274101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:28.454616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934bea4-c0df-4d62-99c1-cd1492274101 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:28.454932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934bea4-c0df-4d62-99c1-cd1492274101 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:28.455208 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934bea4-c0df-4d62-99c1-cd1492274101 None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:28.455829 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934bea4-c0df-4d62-99c1-cd1492274101 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:28.456194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934bea4-c0df-4d62-99c1-cd1492274101 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:28.457572 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934bea4-c0df-4d62-99c1-cd1492274101 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:28.464651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 489/987] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 5 14:15:28 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:28.468239 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f246b4a-59a0-42f8-9fba-bd74a80dd309 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:28.573295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f246b4a-59a0-42f8-9fba-bd74a80dd309 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:28.607659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedb7b4-afe4-47c4-9b8b-9206f2d76975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:28.608274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedb7b4-afe4-47c4-9b8b-9206f2d76975 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:28.608550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedb7b4-afe4-47c4-9b8b-9206f2d76975 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:28.608820 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedb7b4-afe4-47c4-9b8b-9206f2d76975 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:28.609508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cedb7b4-afe4-47c4-9b8b-9206f2d76975 None admin] RBAC: Adding query filter params (interface=public, service_id=6e72434f47484c3eabf04dae38d82e09, region_id=RegionOne) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:28.609810 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cedb7b4-afe4-47c4-9b8b-9206f2d76975 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:28.611183 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cedb7b4-afe4-47c4-9b8b-9206f2d76975 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:28.621280 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 499/988] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Sep 5 14:15:28 2022] GET /identity/v3/endpoints?service_id=6e72434f47484c3eabf04dae38d82e09&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:28.622929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934bea4-c0df-4d62-99c1-cd1492274101 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:28.724367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934bea4-c0df-4d62-99c1-cd1492274101 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:28.753316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:28.758887 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f2345-3561-4654-8760-f408745f943f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:28.759496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f2345-3561-4654-8760-f408745f943f None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:28.759779 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f2345-3561-4654-8760-f408745f943f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:28.760050 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f2345-3561-4654-8760-f408745f943f None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:28.760647 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f2345-3561-4654-8760-f408745f943f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:28.762108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f2345-3561-4654-8760-f408745f943f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:28.768305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 490/989] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 5 14:15:28 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:15:29.027767 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79738d6a-0d98-4442-a4b1-92f7601b00ae None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:29.027957 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79738d6a-0d98-4442-a4b1-92f7601b00ae None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:29.028249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79738d6a-0d98-4442-a4b1-92f7601b00ae None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:29.029720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 500/990] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:29.034781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b9cbf-ec7c-4f45-a1db-118b1481c66f None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:29.035103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b9cbf-ec7c-4f45-a1db-118b1481c66f None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:29.035388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b9cbf-ec7c-4f45-a1db-118b1481c66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:29.054379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-da6b9cbf-ec7c-4f45-a1db-118b1481c66f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:29.056128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-da6b9cbf-ec7c-4f45-a1db-118b1481c66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:29.138871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 491/991] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:29.405190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79738d6a-0d98-4442-a4b1-92f7601b00ae None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:29.469086 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:29.508802 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79738d6a-0d98-4442-a4b1-92f7601b00ae None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:29.543668 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230da21-c83b-489e-8972-093ab1bd6f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:29.544310 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230da21-c83b-489e-8972-093ab1bd6f1d None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:29.544592 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230da21-c83b-489e-8972-093ab1bd6f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:29.544863 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230da21-c83b-489e-8972-093ab1bd6f1d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:29.545434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230da21-c83b-489e-8972-093ab1bd6f1d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:29.546820 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230da21-c83b-489e-8972-093ab1bd6f1d None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:29.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-4230da21-c83b-489e-8972-093ab1bd6f1d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 14:15:29.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:29.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:29.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:29.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:29.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:29.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:29.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:29.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 14:15:29.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:29.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:29.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:29.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:29.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:29.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:29.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:29.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:29.557151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:29.557151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:29.557151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:29.557151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:29.557151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:29.557151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:29.557151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:29.557151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:29.557151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:29.557151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:29.557151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:29.557151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:29.557151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:29.557151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:29.557151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:29.557151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:29.559726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:29.559726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:29.559726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:29.559726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:29.559726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:29.559726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:29.559726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:29.559726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:29.559726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:29.559726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:29.559726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:29.559726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:29.559726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 14:15:29.559726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:29.559726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 501/992] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 5 14:15:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:29.563632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b9cbf-ec7c-4f45-a1db-118b1481c66f None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:29.663301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b9cbf-ec7c-4f45-a1db-118b1481c66f None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:29.688773 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:29.698537 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ba1f9-59a9-47ae-9202-713d3f314dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:29.703604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ba1f9-59a9-47ae-9202-713d3f314dcc None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:29.703604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ba1f9-59a9-47ae-9202-713d3f314dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:29.703604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ba1f9-59a9-47ae-9202-713d3f314dcc None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:29.703604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ba1f9-59a9-47ae-9202-713d3f314dcc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:29.703604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ba1f9-59a9-47ae-9202-713d3f314dcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:29.703604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ba1f9-59a9-47ae-9202-713d3f314dcc None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:29.710854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 492/993] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 5 14:15:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:29.713604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230da21-c83b-489e-8972-093ab1bd6f1d None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:29.817994 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230da21-c83b-489e-8972-093ab1bd6f1d None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:29.845090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:29.854922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decc978-2e13-4f9e-9c1a-116f448f288a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:29.858067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decc978-2e13-4f9e-9c1a-116f448f288a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:29.858067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decc978-2e13-4f9e-9c1a-116f448f288a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:29.858067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decc978-2e13-4f9e-9c1a-116f448f288a None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:29.858067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decc978-2e13-4f9e-9c1a-116f448f288a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:29.858067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decc978-2e13-4f9e-9c1a-116f448f288a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:29.858461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decc978-2e13-4f9e-9c1a-116f448f288a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:29.868418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 502/994] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 5 14:15:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:29.870180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ba1f9-59a9-47ae-9202-713d3f314dcc None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:29.930681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6046f2-3cd5-41e6-be4e-978acfe7c73b None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:29.931007 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6046f2-3cd5-41e6-be4e-978acfe7c73b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:29.931191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6046f2-3cd5-41e6-be4e-978acfe7c73b None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:29.932743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 503/995] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:15:29.937302 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b975d4-ed7a-4ecf-a9b4-581634ebaca4 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:29.937584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b975d4-ed7a-4ecf-a9b4-581634ebaca4 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:29.937855 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b975d4-ed7a-4ecf-a9b4-581634ebaca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:29.957107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-c7b975d4-ed7a-4ecf-a9b4-581634ebaca4 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:29.958306 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c7b975d4-ed7a-4ecf-a9b4-581634ebaca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:29.979632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ba1f9-59a9-47ae-9202-713d3f314dcc None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:30.015318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aae55a-2df0-401d-95a2-2f97eb9f7e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:30.015921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aae55a-2df0-401d-95a2-2f97eb9f7e29 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:30.016205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aae55a-2df0-401d-95a2-2f97eb9f7e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:30.016480 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aae55a-2df0-401d-95a2-2f97eb9f7e29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:30.017181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aae55a-2df0-401d-95a2-2f97eb9f7e29 None admin] RBAC: Adding query filter params (interface=admin, service_id=ffb5ded7bdf14bfd8cea4051e9eddcb8, region_id=RegionOne) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:30.017489 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aae55a-2df0-401d-95a2-2f97eb9f7e29 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:30.018875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aae55a-2df0-401d-95a2-2f97eb9f7e29 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:30.028887 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 493/996] 10.222.0.15 () {56 vars in 1249 bytes} [Mon Sep 5 14:15:29 2022] GET /identity/v3/endpoints?service_id=ffb5ded7bdf14bfd8cea4051e9eddcb8&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:30.029903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aae55a-2df0-401d-95a2-2f97eb9f7e29 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:30.038516 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 504/997] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:30.096131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:30.138663 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aae55a-2df0-401d-95a2-2f97eb9f7e29 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:30.173528 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da476bf-cb9c-4602-becf-03f0117c3d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:30.174238 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da476bf-cb9c-4602-becf-03f0117c3d58 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:30.174495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da476bf-cb9c-4602-becf-03f0117c3d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:30.174769 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da476bf-cb9c-4602-becf-03f0117c3d58 None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:30.175445 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da476bf-cb9c-4602-becf-03f0117c3d58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:30.176821 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da476bf-cb9c-4602-becf-03f0117c3d58 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:30.184124 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 494/998] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 5 14:15:30 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:15:30.313959 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b975d4-ed7a-4ecf-a9b4-581634ebaca4 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:30.419970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b975d4-ed7a-4ecf-a9b4-581634ebaca4 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:30.440622 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:30.454928 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6690a1-0d39-4aaa-a9b5-e492dde81884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:30.455573 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6690a1-0d39-4aaa-a9b5-e492dde81884 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:30.455855 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6690a1-0d39-4aaa-a9b5-e492dde81884 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:30.456129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6690a1-0d39-4aaa-a9b5-e492dde81884 None admin] PATH_INFO: `/v3/services/image` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:30.456711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6690a1-0d39-4aaa-a9b5-e492dde81884 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:30.458081 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6690a1-0d39-4aaa-a9b5-e492dde81884 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:30.467453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-7b6690a1-0d39-4aaa-a9b5-e492dde81884 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 14:15:30.467453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:30.467453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:30.467453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:30.467453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:30.467453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:30.467453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:30.467453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:30.467453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 14:15:30.467453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:30.467453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:30.467453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:30.467453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:30.467453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:30.467453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:30.467453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:30.467453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:30.468493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:30.468493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:30.468493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:30.468493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:30.468493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:30.468493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:30.468493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:30.468493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:30.468493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:30.468493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:30.468493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:30.468493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:30.468493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:30.468493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:30.468493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:30.468493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:30.469371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:30.469371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:30.469371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:30.469371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:30.469371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:30.469371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:30.469371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:30.469371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:30.469371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:30.469371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:30.469371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:30.469371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:30.469371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 14:15:30.469371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:30.469371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 505/999] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Sep 5 14:15:30 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:30.476093 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da476bf-cb9c-4602-becf-03f0117c3d58 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:30.576909 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da476bf-cb9c-4602-becf-03f0117c3d58 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:30.594295 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:30.611017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ccbd2-a1fb-4f46-9f1e-a2ba9b6bb61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:30.611628 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ccbd2-a1fb-4f46-9f1e-a2ba9b6bb61e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:30.611905 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ccbd2-a1fb-4f46-9f1e-a2ba9b6bb61e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:30.612175 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ccbd2-a1fb-4f46-9f1e-a2ba9b6bb61e None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:30.612792 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ccbd2-a1fb-4f46-9f1e-a2ba9b6bb61e None admin] RBAC: Adding query filter params (name=image) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:30.613090 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ccbd2-a1fb-4f46-9f1e-a2ba9b6bb61e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:30.614504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ccbd2-a1fb-4f46-9f1e-a2ba9b6bb61e None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:30.624509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 495/1000] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:30.626562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6690a1-0d39-4aaa-a9b5-e492dde81884 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:30.678179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa27f7e-e1a1-480b-9141-05fbf0157aed None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:30.678699 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa27f7e-e1a1-480b-9141-05fbf0157aed None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:30.678995 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa27f7e-e1a1-480b-9141-05fbf0157aed None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:30.680625 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 496/1001] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:30.687063 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f288256-c0c5-4031-b2aa-8b8eed065b45 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:30.687360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f288256-c0c5-4031-b2aa-8b8eed065b45 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:30.687623 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f288256-c0c5-4031-b2aa-8b8eed065b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:30.706969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-8f288256-c0c5-4031-b2aa-8b8eed065b45 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:30.708196 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8f288256-c0c5-4031-b2aa-8b8eed065b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:30.736098 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6690a1-0d39-4aaa-a9b5-e492dde81884 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:30.769985 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6b2e3-e3db-46fc-9cc9-3e1f5b8febb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:30.770589 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6b2e3-e3db-46fc-9cc9-3e1f5b8febb6 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:30.770899 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6b2e3-e3db-46fc-9cc9-3e1f5b8febb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:30.771177 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6b2e3-e3db-46fc-9cc9-3e1f5b8febb6 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:30.771862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6b2e3-e3db-46fc-9cc9-3e1f5b8febb6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:30.772172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6b2e3-e3db-46fc-9cc9-3e1f5b8febb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:30.773562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6b2e3-e3db-46fc-9cc9-3e1f5b8febb6 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:30.778371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 506/1002] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:30 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:30.782836 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 497/1003] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:31.050191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6b2e3-e3db-46fc-9cc9-3e1f5b8febb6 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:31.104763 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:31.156157 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6b2e3-e3db-46fc-9cc9-3e1f5b8febb6 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:31.189920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2776d8d0-553b-4ba5-90e3-0f21af50b66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:31.190559 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776d8d0-553b-4ba5-90e3-0f21af50b66f None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:31.190933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776d8d0-553b-4ba5-90e3-0f21af50b66f None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:31.191213 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776d8d0-553b-4ba5-90e3-0f21af50b66f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:31.192058 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2776d8d0-553b-4ba5-90e3-0f21af50b66f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:31.193428 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2776d8d0-553b-4ba5-90e3-0f21af50b66f None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:31.202216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-2776d8d0-553b-4ba5-90e3-0f21af50b66f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 14:15:31.202216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:31.202216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:31.202216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:31.202216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:31.202216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:31.202216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:31.202216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:31.202216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 14:15:31.202216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:31.202216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:31.202216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:31.202216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:31.202216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:31.202216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:31.202216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:31.202216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:31.203487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:31.203487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:31.203487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:31.203487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:31.203487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:31.203487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:31.203487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:31.203487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:31.203487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:31.203487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:31.203487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:31.203487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:31.203487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:31.203487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:31.203487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:31.203487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:31.206605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:31.206605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:31.206605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:31.206605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:31.206605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:31.206605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:31.206605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:31.206605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:31.206605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:31.206605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:31.206605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:31.206605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:31.206605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 14:15:31.206605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:31.207722 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 507/1004] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:31.209298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f288256-c0c5-4031-b2aa-8b8eed065b45 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:31.313594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f288256-c0c5-4031-b2aa-8b8eed065b45 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:31.327240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:31.347492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f282fac-44cb-45b3-b8a3-9b943ee31fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:31.348157 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f282fac-44cb-45b3-b8a3-9b943ee31fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:31.348432 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f282fac-44cb-45b3-b8a3-9b943ee31fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:31.348699 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f282fac-44cb-45b3-b8a3-9b943ee31fe0 None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:31.349316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f282fac-44cb-45b3-b8a3-9b943ee31fe0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:31.349621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f282fac-44cb-45b3-b8a3-9b943ee31fe0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:31.351029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f282fac-44cb-45b3-b8a3-9b943ee31fe0 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:31.360629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 498/1005] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 5 14:15:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:31.362094 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2776d8d0-553b-4ba5-90e3-0f21af50b66f None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:31.461801 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2776d8d0-553b-4ba5-90e3-0f21af50b66f None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:31.474150 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:31.496842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c23df0-f729-4d7b-8f6b-b4188df63783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:31.497450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c23df0-f729-4d7b-8f6b-b4188df63783 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:31.497730 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c23df0-f729-4d7b-8f6b-b4188df63783 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:31.498001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c23df0-f729-4d7b-8f6b-b4188df63783 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:31.498625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c23df0-f729-4d7b-8f6b-b4188df63783 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:31.498954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c23df0-f729-4d7b-8f6b-b4188df63783 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:31.500317 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c23df0-f729-4d7b-8f6b-b4188df63783 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:31.505380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 508/1006] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 5 14:15:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:31.511021 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f282fac-44cb-45b3-b8a3-9b943ee31fe0 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:31.609797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f282fac-44cb-45b3-b8a3-9b943ee31fe0 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:31.643794 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174fc7e-1f4b-4cda-abdc-acecddfe8260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:31.644458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174fc7e-1f4b-4cda-abdc-acecddfe8260 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:31.644745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174fc7e-1f4b-4cda-abdc-acecddfe8260 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:31.645020 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174fc7e-1f4b-4cda-abdc-acecddfe8260 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:31.645686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174fc7e-1f4b-4cda-abdc-acecddfe8260 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:31.647140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174fc7e-1f4b-4cda-abdc-acecddfe8260 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:31.758946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 499/1007] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:15:31 2022] POST /identity/v3/endpoints => generated 334 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:15:32.111639 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd5731-24e6-46bf-aae9-b8a2b7ab3d8f None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:32.111931 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd5731-24e6-46bf-aae9-b8a2b7ab3d8f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:32.112205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd5731-24e6-46bf-aae9-b8a2b7ab3d8f None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:32.113687 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 509/1008] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:32.119134 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019e241-9915-4f27-ba26-4ad5fadbf32a None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:32.119425 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019e241-9915-4f27-ba26-4ad5fadbf32a None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:32.119703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019e241-9915-4f27-ba26-4ad5fadbf32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:32.137689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-9019e241-9915-4f27-ba26-4ad5fadbf32a None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:32.138904 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9019e241-9915-4f27-ba26-4ad5fadbf32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:32.165907 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:32.213493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 500/1009] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:32 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:32.479027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd5731-24e6-46bf-aae9-b8a2b7ab3d8f None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:32.579940 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd5731-24e6-46bf-aae9-b8a2b7ab3d8f None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:32.612862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb148caa-e89f-48fe-903c-18c2ff404da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:32.613486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb148caa-e89f-48fe-903c-18c2ff404da3 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:32.613764 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb148caa-e89f-48fe-903c-18c2ff404da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:32.614034 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb148caa-e89f-48fe-903c-18c2ff404da3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:32.614603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb148caa-e89f-48fe-903c-18c2ff404da3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:32.615978 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb148caa-e89f-48fe-903c-18c2ff404da3 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:32.625049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-bb148caa-e89f-48fe-903c-18c2ff404da3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 14:15:32.625049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:32.625049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:32.625049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:32.625049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:32.625049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:32.625049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:32.625049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:32.625049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 14:15:32.625049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:32.625049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:32.625049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:32.625049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:32.625049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:32.625049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:32.625049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:32.625049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:32.626047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:32.626047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:32.626047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:32.626047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:32.626047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:32.626047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:32.626047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:32.626047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:32.626047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:32.626047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:32.626047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:32.626047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:32.626047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:32.626047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:32.626047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:32.626047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:32.627409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:32.627409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:32.627409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:32.627409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:32.627409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:32.627409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:32.627409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:32.627409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:32.627409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:32.627409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:32.627409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:32.627409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:32.627409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 14:15:32.627409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:32.628560 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 510/1010] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 5 14:15:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:32.631971 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019e241-9915-4f27-ba26-4ad5fadbf32a None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:32.681294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbcdd4-4b71-4dd4-9414-3ba99dabac6d None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:32.681661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbcdd4-4b71-4dd4-9414-3ba99dabac6d None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:32.681861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbcdd4-4b71-4dd4-9414-3ba99dabac6d None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:32.683233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 511/1011] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:15:32.688377 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c4858-fd5c-4b9e-93c0-8f744cb2af32 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:32.688658 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c4858-fd5c-4b9e-93c0-8f744cb2af32 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:32.688933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c4858-fd5c-4b9e-93c0-8f744cb2af32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:32.709123 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-773c4858-fd5c-4b9e-93c0-8f744cb2af32 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:32.710303 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-773c4858-fd5c-4b9e-93c0-8f744cb2af32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:32.739551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019e241-9915-4f27-ba26-4ad5fadbf32a None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:32.746182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:32.775205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145874f4-c428-4bf1-a0af-8e14f90be0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:32.775840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145874f4-c428-4bf1-a0af-8e14f90be0d6 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:32.776126 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145874f4-c428-4bf1-a0af-8e14f90be0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:32.776402 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145874f4-c428-4bf1-a0af-8e14f90be0d6 None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:32.777056 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145874f4-c428-4bf1-a0af-8e14f90be0d6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:32.777365 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145874f4-c428-4bf1-a0af-8e14f90be0d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:32.778776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145874f4-c428-4bf1-a0af-8e14f90be0d6 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:32.781506 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 512/1012] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:32 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:32.788609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 501/1013] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 5 14:15:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:32.791773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c4858-fd5c-4b9e-93c0-8f744cb2af32 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:32.894220 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c4858-fd5c-4b9e-93c0-8f744cb2af32 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:32.931289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd5587-ae40-4e4c-b045-455ed47bbdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:32.931908 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd5587-ae40-4e4c-b045-455ed47bbdef None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:32.932191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd5587-ae40-4e4c-b045-455ed47bbdef None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:32.932465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd5587-ae40-4e4c-b045-455ed47bbdef None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:32.933094 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd5587-ae40-4e4c-b045-455ed47bbdef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:32.933399 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd5587-ae40-4e4c-b045-455ed47bbdef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:32.934763 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd5587-ae40-4e4c-b045-455ed47bbdef None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:32.939755 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 513/1014] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 5 14:15:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:32.945495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145874f4-c428-4bf1-a0af-8e14f90be0d6 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:32.970905 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:33.048320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145874f4-c428-4bf1-a0af-8e14f90be0d6 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:33.051669 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd5587-ae40-4e4c-b045-455ed47bbdef None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:33.085344 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64414026-820f-4151-a23b-99727e9e782f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:33.086480 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64414026-820f-4151-a23b-99727e9e782f None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:33.087105 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64414026-820f-4151-a23b-99727e9e782f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:33.087384 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64414026-820f-4151-a23b-99727e9e782f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:33.088096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64414026-820f-4151-a23b-99727e9e782f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:33.089462 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64414026-820f-4151-a23b-99727e9e782f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:33.093127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:33.161195 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd5587-ae40-4e4c-b045-455ed47bbdef None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:33.197079 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fb4b5-4261-4fb3-b534-b996d4f03043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:33.197713 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fb4b5-4261-4fb3-b534-b996d4f03043 None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:33.197965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fb4b5-4261-4fb3-b534-b996d4f03043 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:33.198238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fb4b5-4261-4fb3-b534-b996d4f03043 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:33.198903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fb4b5-4261-4fb3-b534-b996d4f03043 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:33.200266 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fb4b5-4261-4fb3-b534-b996d4f03043 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:33.211077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 502/1015] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:15:32 2022] POST /identity/v3/endpoints => generated 325 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:15:33.275663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 514/1016] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 5 14:15:33 2022] POST /identity/v3/services => generated 242 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:15:33.761762 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7069fcf-d115-4552-a43c-48e9c80d5f69 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:33.762300 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7069fcf-d115-4552-a43c-48e9c80d5f69 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:33.762300 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7069fcf-d115-4552-a43c-48e9c80d5f69 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:33.763802 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 503/1017] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:33.768656 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6abb62-116e-4bbf-8a5c-4c02b86fa140 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:33.769053 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6abb62-116e-4bbf-8a5c-4c02b86fa140 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:33.769270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6abb62-116e-4bbf-8a5c-4c02b86fa140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:33.786918 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-8a6abb62-116e-4bbf-8a5c-4c02b86fa140 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:33.788094 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8a6abb62-116e-4bbf-8a5c-4c02b86fa140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:33.860023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 515/1018] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:34.128229 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7069fcf-d115-4552-a43c-48e9c80d5f69 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:34.212361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:34.231726 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7069fcf-d115-4552-a43c-48e9c80d5f69 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:34.266833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ed743-04a9-4ec2-b240-4c05956953dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:34.267156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ed743-04a9-4ec2-b240-4c05956953dd None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:34.267356 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ed743-04a9-4ec2-b240-4c05956953dd None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:34.267616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ed743-04a9-4ec2-b240-4c05956953dd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:34.279201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ed743-04a9-4ec2-b240-4c05956953dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:15:34.283972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-d75ed743-04a9-4ec2-b240-4c05956953dd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 14:15:34.283972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:34.283972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:34.283972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:34.283972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:34.283972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:34.283972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:34.283972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:34.283972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:34.283972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:34.283972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:34.283972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:34.283972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:15:34.283972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:15:34.283972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:15:34.283972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:15:34.285051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:15:34.285051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:15:34.285051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:15:34.285051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:15:34.285051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:34.285051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:34.285051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:34.285051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:34.285051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:34.285051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:34.285051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:34.285051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:34.285051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:34.285051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:34.285051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:34.285051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:34.285955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:34.285955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:34.285955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:34.285955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:34.285955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:15:34.285955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:15:34.285955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:15:34.285955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:15:34.285955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:15:34.285955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:15:34.285955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 14:15:34.285955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:34.287412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 504/1019] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Sep 5 14:15:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:34.291575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6abb62-116e-4bbf-8a5c-4c02b86fa140 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:34.314496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:34.396854 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6abb62-116e-4bbf-8a5c-4c02b86fa140 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:34.431544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc6725-b0ea-4ac2-bcc7-6fb4a585c132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:34.432140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc6725-b0ea-4ac2-bcc7-6fb4a585c132 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:34.432418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc6725-b0ea-4ac2-bcc7-6fb4a585c132 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:34.432685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc6725-b0ea-4ac2-bcc7-6fb4a585c132 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:34.433347 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc6725-b0ea-4ac2-bcc7-6fb4a585c132 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:34.433632 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc6725-b0ea-4ac2-bcc7-6fb4a585c132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:34.435002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc6725-b0ea-4ac2-bcc7-6fb4a585c132 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:34.443394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 516/1020] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Sep 5 14:15:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:34.449316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ed743-04a9-4ec2-b240-4c05956953dd None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:34.549181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ed743-04a9-4ec2-b240-4c05956953dd None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:34.582805 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9218a-6a58-4c8b-bb21-aeb6c8bc3483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:34.583516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9218a-6a58-4c8b-bb21-aeb6c8bc3483 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:34.583798 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9218a-6a58-4c8b-bb21-aeb6c8bc3483 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:34.584070 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9218a-6a58-4c8b-bb21-aeb6c8bc3483 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:34.593977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9218a-6a58-4c8b-bb21-aeb6c8bc3483 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:34.595523 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9218a-6a58-4c8b-bb21-aeb6c8bc3483 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:34.597194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:34.611734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-07d9218a-6a58-4c8b-bb21-aeb6c8bc3483 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:34.611734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:34.611734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:34.611734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:34.611734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:34.611734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:34.611734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:34.611734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:34.611734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:34.611734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:34.611734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:34.611734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:34.611734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:34.611734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:34.611734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:34.611734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:34.612741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:34.612741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:34.612741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:34.612741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:34.612741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:34.612741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:34.612741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:34.612741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:34.612741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:34.612741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:34.612741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:34.612741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:34.612741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:34.612741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:34.612741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:34.612741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:34.612741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:34.613804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:34.613804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:34.613804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:34.613804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 505/1021] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:34.620894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc6725-b0ea-4ac2-bcc7-6fb4a585c132 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:34.704795 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:34.723448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc6725-b0ea-4ac2-bcc7-6fb4a585c132 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:34.758211 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d77336-07db-48a6-aba4-1ca2f851a734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:34.758835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d77336-07db-48a6-aba4-1ca2f851a734 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:34.759118 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d77336-07db-48a6-aba4-1ca2f851a734 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:34.759390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d77336-07db-48a6-aba4-1ca2f851a734 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:34.760019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d77336-07db-48a6-aba4-1ca2f851a734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:34.760320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d77336-07db-48a6-aba4-1ca2f851a734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:34.761678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d77336-07db-48a6-aba4-1ca2f851a734 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:34.773541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 517/1022] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:34.779505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9218a-6a58-4c8b-bb21-aeb6c8bc3483 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:34.879247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9218a-6a58-4c8b-bb21-aeb6c8bc3483 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:34.913554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0875cd7-a343-4bd3-bb68-a1dcd551c164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:34.914180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0875cd7-a343-4bd3-bb68-a1dcd551c164 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:34.914510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0875cd7-a343-4bd3-bb68-a1dcd551c164 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:34.914772 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0875cd7-a343-4bd3-bb68-a1dcd551c164 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:34.922598 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0875cd7-a343-4bd3-bb68-a1dcd551c164 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:34.924120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0875cd7-a343-4bd3-bb68-a1dcd551c164 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:34.935566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-d0875cd7-a343-4bd3-bb68-a1dcd551c164 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:15:34.935566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:34.935566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:34.935566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:34.935566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:34.935566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:34.935566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:34.935566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:34.935566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:34.935566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:34.935566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:34.935566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:34.935566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:15:34.935566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:15:34.935566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:15:34.935566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:34.936547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:34.937511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:34.937511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:34.937511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:15:34.937511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:15:34.937511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:15:34.937511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:15:34.937511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:15:34.937511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:15:34.937511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:15:34.937511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:34.939017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 506/1023] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 5 14:15:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:34.942668 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d77336-07db-48a6-aba4-1ca2f851a734 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:35.044403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d77336-07db-48a6-aba4-1ca2f851a734 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:35.083912 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608efa48-ffeb-4acf-9acd-d2415e86ff55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:35.084578 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608efa48-ffeb-4acf-9acd-d2415e86ff55 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:35.084859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608efa48-ffeb-4acf-9acd-d2415e86ff55 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:35.085188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608efa48-ffeb-4acf-9acd-d2415e86ff55 None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:35.085928 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608efa48-ffeb-4acf-9acd-d2415e86ff55 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:35.086764 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608efa48-ffeb-4acf-9acd-d2415e86ff55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:35.088213 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608efa48-ffeb-4acf-9acd-d2415e86ff55 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:35.105292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:35.113532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 518/1024] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 5 14:15:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:35.115091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0875cd7-a343-4bd3-bb68-a1dcd551c164 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:35.125883 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b51194-1b98-46e9-9e65-4a9aca22dce0 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:35.132588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b51194-1b98-46e9-9e65-4a9aca22dce0 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:35.132588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b51194-1b98-46e9-9e65-4a9aca22dce0 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:35.132588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 519/1025] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:15:35.132588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492d337-5210-4314-9a74-bfac9398b456 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:35.132891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492d337-5210-4314-9a74-bfac9398b456 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:35.133672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492d337-5210-4314-9a74-bfac9398b456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:35.151470 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-7492d337-5210-4314-9a74-bfac9398b456 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:35.152655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7492d337-5210-4314-9a74-bfac9398b456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:35.175310 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:35.222937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 520/1026] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:35.227799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0875cd7-a343-4bd3-bb68-a1dcd551c164 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:35.235868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8ea39-94b3-48cd-8cc6-0d38776083b1 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:35.236157 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8ea39-94b3-48cd-8cc6-0d38776083b1 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:35.236432 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8ea39-94b3-48cd-8cc6-0d38776083b1 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:35.238253 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 521/1027] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:15:35.243622 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ffd78-cef3-4eb7-9036-509a4ad37257 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:35.243905 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ffd78-cef3-4eb7-9036-509a4ad37257 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:35.244177 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ffd78-cef3-4eb7-9036-509a4ad37257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:35.263235 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-556ffd78-cef3-4eb7-9036-509a4ad37257 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:35.263412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b02b72-684c-4cbe-a96b-df439a837f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:35.264031 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b02b72-684c-4cbe-a96b-df439a837f7a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:35.264309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b02b72-684c-4cbe-a96b-df439a837f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:35.264579 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b02b72-684c-4cbe-a96b-df439a837f7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:35.265409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-556ffd78-cef3-4eb7-9036-509a4ad37257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:35.334250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b02b72-684c-4cbe-a96b-df439a837f7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:35.335660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b02b72-684c-4cbe-a96b-df439a837f7a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:35.336737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 522/1028] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:35.392891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:35.437291 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 507/1029] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:15:35 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:15:35.443103 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ffd78-cef3-4eb7-9036-509a4ad37257 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:35.488246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b02b72-684c-4cbe-a96b-df439a837f7a None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:35.502038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:35.545906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ffd78-cef3-4eb7-9036-509a4ad37257 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:35.581884 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a1392-dc50-4435-8e31-6fdb012d1433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:35.582740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a1392-dc50-4435-8e31-6fdb012d1433 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:35.583053 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a1392-dc50-4435-8e31-6fdb012d1433 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:35.583327 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a1392-dc50-4435-8e31-6fdb012d1433 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:35.593602 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b02b72-684c-4cbe-a96b-df439a837f7a None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:35.594517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a1392-dc50-4435-8e31-6fdb012d1433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:35.596007 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a1392-dc50-4435-8e31-6fdb012d1433 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:35.611023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-af6a1392-dc50-4435-8e31-6fdb012d1433 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:35.611023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:35.611023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:35.611023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:35.611023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:35.611023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:35.611023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:35.611023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:35.611023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:35.611023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:35.611023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:35.611023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:35.611023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:35.611023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:35.611023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:35.611023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:35.612004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:35.612004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:35.612004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:35.612004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:35.612004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:35.612004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:35.612004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:35.612004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:35.612004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:35.612004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:35.612004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:35.612004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:35.612004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:35.612004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:35.612004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:35.612004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:35.612004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:35.612922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:35.612922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:35.612922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:35.612922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 523/1030] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:35.613703 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a1392-dc50-4435-8e31-6fdb012d1433 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:35.628631 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192efa2-f196-4a80-bf53-44213e9b0a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:35.629262 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192efa2-f196-4a80-bf53-44213e9b0a08 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:35.629541 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192efa2-f196-4a80-bf53-44213e9b0a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:35.629813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192efa2-f196-4a80-bf53-44213e9b0a08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:35.630917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192efa2-f196-4a80-bf53-44213e9b0a08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:35.632331 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192efa2-f196-4a80-bf53-44213e9b0a08 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:35.638387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 508/1031] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:35.638387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192efa2-f196-4a80-bf53-44213e9b0a08 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:35.662830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:35.717360 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a1392-dc50-4435-8e31-6fdb012d1433 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:35.730025 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:35.737432 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192efa2-f196-4a80-bf53-44213e9b0a08 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:35.750593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668de40-da80-43b4-aae8-06f1209c07ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:35.751249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668de40-da80-43b4-aae8-06f1209c07ff None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:35.751557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668de40-da80-43b4-aae8-06f1209c07ff None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:35.751802 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668de40-da80-43b4-aae8-06f1209c07ff None admin] PATH_INFO: `/v3/services/image` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:35.752378 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668de40-da80-43b4-aae8-06f1209c07ff None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:35.753744 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668de40-da80-43b4-aae8-06f1209c07ff None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:35.762463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-0668de40-da80-43b4-aae8-06f1209c07ff None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 14:15:35.762463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:35.762463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:35.762463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:35.762463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:35.762463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:35.762463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:35.762463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:35.762463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 14:15:35.762463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:35.762463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:35.762463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:35.762463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:35.762463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:35.762463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:35.762463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:35.762463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:35.763545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:35.763545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:35.763545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:35.763545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:35.763545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:35.763545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:35.763545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:35.763545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:35.763545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:35.763545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:35.763545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:35.763545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:35.763545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:35.763545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:35.763545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:35.763545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:35.764861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:35.764861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:35.764861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:35.764861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:35.764861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:35.764861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:35.764861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:35.764861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:35.764861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:35.764861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:35.764861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:35.764861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:35.764861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 14:15:35.764861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:35.764861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 524/1032] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Sep 5 14:15:35 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:35.765865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668de40-da80-43b4-aae8-06f1209c07ff None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:35.774524 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b6cd0-f6f3-488b-a418-cb1d1b2e3a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:35.775161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b6cd0-f6f3-488b-a418-cb1d1b2e3a5c None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:35.775441 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b6cd0-f6f3-488b-a418-cb1d1b2e3a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:35.775710 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b6cd0-f6f3-488b-a418-cb1d1b2e3a5c None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:35.776339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b6cd0-f6f3-488b-a418-cb1d1b2e3a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:35.776639 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b6cd0-f6f3-488b-a418-cb1d1b2e3a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:35.777989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b6cd0-f6f3-488b-a418-cb1d1b2e3a5c None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:35.789262 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 509/1033] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:35.789939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b6cd0-f6f3-488b-a418-cb1d1b2e3a5c None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:35.869499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:35.871053 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668de40-da80-43b4-aae8-06f1209c07ff None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:35.887513 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b6cd0-f6f3-488b-a418-cb1d1b2e3a5c None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:35.905631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b80b78-fce8-4ccd-8a27-a81b278c9f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:35.906237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b80b78-fce8-4ccd-8a27-a81b278c9f46 None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:35.906517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b80b78-fce8-4ccd-8a27-a81b278c9f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:35.906815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b80b78-fce8-4ccd-8a27-a81b278c9f46 None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:35.907457 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b80b78-fce8-4ccd-8a27-a81b278c9f46 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:35.908883 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b80b78-fce8-4ccd-8a27-a81b278c9f46 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:35.920681 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb0eb0-a0d1-4436-b14b-bbcec5b81ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:35.921269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb0eb0-a0d1-4436-b14b-bbcec5b81ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:35.921557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb0eb0-a0d1-4436-b14b-bbcec5b81ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:35.921825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb0eb0-a0d1-4436-b14b-bbcec5b81ae6 None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:35.922444 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb0eb0-a0d1-4436-b14b-bbcec5b81ae6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:35.922742 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb0eb0-a0d1-4436-b14b-bbcec5b81ae6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:35.924131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb0eb0-a0d1-4436-b14b-bbcec5b81ae6 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:35.928840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 510/1034] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:35.977577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb0eb0-a0d1-4436-b14b-bbcec5b81ae6 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:36.028721 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb0eb0-a0d1-4436-b14b-bbcec5b81ae6 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:36.060574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f74cf-24e8-48c8-918b-b059f30798d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:36.061179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f74cf-24e8-48c8-918b-b059f30798d5 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:36.061461 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f74cf-24e8-48c8-918b-b059f30798d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:36.061733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f74cf-24e8-48c8-918b-b059f30798d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:36.068405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f74cf-24e8-48c8-918b-b059f30798d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:36.069824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f74cf-24e8-48c8-918b-b059f30798d5 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:36.082203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-f53f74cf-24e8-48c8-918b-b059f30798d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:36.082203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:36.082203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:36.082203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:36.082203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:36.082203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:36.082203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:36.082203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:36.082203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:36.082203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:36.082203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:36.082203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:36.082203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:15:36.082203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:15:36.082203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:15:36.083435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:15:36.083435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:36.083435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:36.083435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:36.083435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:36.083435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:36.083435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:36.083435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:36.083435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:36.083435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:36.083435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:36.083435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:36.083435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:36.083435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:36.083435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:36.083435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:36.083435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:36.084568 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:15:36.084568 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:15:36.084568 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:15:36.084568 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:15:36.084568 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:15:36.084568 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:15:36.084568 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:36.084568 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:36.085068 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 525/1035] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 5 14:15:35 2022] POST /identity/v3/projects => generated 311 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:15:36.086988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 511/1036] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:36.087949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f74cf-24e8-48c8-918b-b059f30798d5 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:36.090149 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b80b78-fce8-4ccd-8a27-a81b278c9f46 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:36.097785 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:36.127151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:36.203697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f74cf-24e8-48c8-918b-b059f30798d5 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:36.208733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b80b78-fce8-4ccd-8a27-a81b278c9f46 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:36.244529 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9988479-360e-483d-b795-02f45e19c348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:36.245779 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9988479-360e-483d-b795-02f45e19c348 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:36.245779 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9988479-360e-483d-b795-02f45e19c348 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:36.245779 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9988479-360e-483d-b795-02f45e19c348 None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:36.246295 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9988479-360e-483d-b795-02f45e19c348 None admin] RBAC: Adding query filter params (type=image) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:36.246600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9988479-360e-483d-b795-02f45e19c348 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:36.248001 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9988479-360e-483d-b795-02f45e19c348 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:36.254310 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 512/1037] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:36.257029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7eb67-a20c-4700-a5a6-1446f5ab2588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:36.257657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7eb67-a20c-4700-a5a6-1446f5ab2588 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:36.257939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7eb67-a20c-4700-a5a6-1446f5ab2588 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:36.258215 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7eb67-a20c-4700-a5a6-1446f5ab2588 None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:36.258995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7eb67-a20c-4700-a5a6-1446f5ab2588 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:36.259309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7eb67-a20c-4700-a5a6-1446f5ab2588 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:36.259772 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9988479-360e-483d-b795-02f45e19c348 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:36.260746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7eb67-a20c-4700-a5a6-1446f5ab2588 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:36.275229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 526/1038] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:15:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:36.282550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7eb67-a20c-4700-a5a6-1446f5ab2588 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:36.333113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:36.364785 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9988479-360e-483d-b795-02f45e19c348 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:36.380105 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:36.392636 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7eb67-a20c-4700-a5a6-1446f5ab2588 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:36.402326 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a33ff4-d9ef-4f11-9642-41a349962d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:36.402992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a33ff4-d9ef-4f11-9642-41a349962d86 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:36.403283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a33ff4-d9ef-4f11-9642-41a349962d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:36.403561 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a33ff4-d9ef-4f11-9642-41a349962d86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:36.404256 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a33ff4-d9ef-4f11-9642-41a349962d86 None admin] RBAC: Adding query filter params (interface=public, service_id=42a1956979664ab187b42c7a444ade26, region_id=RegionOne) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:36.404569 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a33ff4-d9ef-4f11-9642-41a349962d86 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:36.405981 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a33ff4-d9ef-4f11-9642-41a349962d86 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:36.411503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 513/1039] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Sep 5 14:15:36 2022] GET /identity/v3/endpoints?service_id=42a1956979664ab187b42c7a444ade26&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:36.416947 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a33ff4-d9ef-4f11-9642-41a349962d86 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:36.429496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0c8be-ff9c-44be-b252-c86f7c52e4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:36.430638 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0c8be-ff9c-44be-b252-c86f7c52e4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:36.430965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0c8be-ff9c-44be-b252-c86f7c52e4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:36.431222 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0c8be-ff9c-44be-b252-c86f7c52e4a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:36.431946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0c8be-ff9c-44be-b252-c86f7c52e4a3 None admin] RBAC: Adding query filter params (role.id=48a9bca3f43c4943855875bf97ec564f, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=709b9ae82dcd47408fc9a6d54a54428a) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:36.432274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0c8be-ff9c-44be-b252-c86f7c52e4a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:36.433668 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0c8be-ff9c-44be-b252-c86f7c52e4a3 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:36.446828 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 527/1040] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:15:36 2022] GET /identity/v3/role_assignments?user.id=709b9ae82dcd47408fc9a6d54a54428a&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=48a9bca3f43c4943855875bf97ec564f => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:36.539120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a33ff4-d9ef-4f11-9642-41a349962d86 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:36.579242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0bf66-a622-41da-b4cb-12c5a7893d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:36.581073 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0bf66-a622-41da-b4cb-12c5a7893d32 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:36.581073 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0bf66-a622-41da-b4cb-12c5a7893d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:36.581073 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0bf66-a622-41da-b4cb-12c5a7893d32 None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:36.581073 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0bf66-a622-41da-b4cb-12c5a7893d32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:36.582364 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0bf66-a622-41da-b4cb-12c5a7893d32 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:36.583542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:36.590665 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 514/1041] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 5 14:15:36 2022] GET /identity/v3/services => generated 1957 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:15:38.021252 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c67015-97f5-4eb3-9eba-8c826941df94 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:38.021806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c67015-97f5-4eb3-9eba-8c826941df94 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:38.021806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c67015-97f5-4eb3-9eba-8c826941df94 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:38.023307 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 528/1042] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:38.028763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26121e6-bee9-4064-8a33-3a329ef0470e None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:38.029151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26121e6-bee9-4064-8a33-3a329ef0470e None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:38.029331 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26121e6-bee9-4064-8a33-3a329ef0470e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:38.047101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-f26121e6-bee9-4064-8a33-3a329ef0470e None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:38.048268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f26121e6-bee9-4064-8a33-3a329ef0470e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:38.115932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 515/1043] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:38.332996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcae1a5-c60d-4140-a148-534078d06a91 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:38.333289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcae1a5-c60d-4140-a148-534078d06a91 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:38.333471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcae1a5-c60d-4140-a148-534078d06a91 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:38.334945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 529/1044] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:38.340302 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e6fce-23ac-4e07-a16d-bf8d2c5a6ac3 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:38.340594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e6fce-23ac-4e07-a16d-bf8d2c5a6ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:38.340871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e6fce-23ac-4e07-a16d-bf8d2c5a6ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:38.358714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-621e6fce-23ac-4e07-a16d-bf8d2c5a6ac3 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:38.359900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-621e6fce-23ac-4e07-a16d-bf8d2c5a6ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:38.395833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcae1a5-c60d-4140-a148-534078d06a91 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:38.427621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 516/1045] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:38.476732 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972b411-a9d3-474a-a19d-77f63ea4a175 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:38.477048 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972b411-a9d3-474a-a19d-77f63ea4a175 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:38.477330 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972b411-a9d3-474a-a19d-77f63ea4a175 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:38.478708 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 517/1046] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:15:38.483476 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af3a9e-8d06-47b4-a1ce-ed7db8d4fc86 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:38.483765 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af3a9e-8d06-47b4-a1ce-ed7db8d4fc86 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:38.484046 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af3a9e-8d06-47b4-a1ce-ed7db8d4fc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:38.501746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcae1a5-c60d-4140-a148-534078d06a91 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:38.501907 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-54af3a9e-8d06-47b4-a1ce-ed7db8d4fc86 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:38.502968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-54af3a9e-8d06-47b4-a1ce-ed7db8d4fc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:38.537072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9cb72-6726-4c6f-9098-4255b7af5692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:38.537697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9cb72-6726-4c6f-9098-4255b7af5692 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:38.537974 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9cb72-6726-4c6f-9098-4255b7af5692 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:38.538241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9cb72-6726-4c6f-9098-4255b7af5692 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:38.539420 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9cb72-6726-4c6f-9098-4255b7af5692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:38.540838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9cb72-6726-4c6f-9098-4255b7af5692 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:38.542535 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 530/1047] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:38.548036 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9cb72-6726-4c6f-9098-4255b7af5692 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:38.562737 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:38.571366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:38.586877 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 518/1048] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:38.647781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9cb72-6726-4c6f-9098-4255b7af5692 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:38.682439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30d72e-c84f-4769-a0bd-1b54ce68fbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:38.683069 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30d72e-c84f-4769-a0bd-1b54ce68fbdd None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:38.683348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30d72e-c84f-4769-a0bd-1b54ce68fbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:38.683617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30d72e-c84f-4769-a0bd-1b54ce68fbdd None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:38.684256 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30d72e-c84f-4769-a0bd-1b54ce68fbdd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:38.685688 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30d72e-c84f-4769-a0bd-1b54ce68fbdd None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:38.695439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af3a9e-8d06-47b4-a1ce-ed7db8d4fc86 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:38.778819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 531/1049] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:15:38 2022] POST /identity/v3/users => generated 285 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:15:38.806195 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af3a9e-8d06-47b4-a1ce-ed7db8d4fc86 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:38.844768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e99be0-f224-4911-9cb3-bed7ff0767e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:38.845334 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e99be0-f224-4911-9cb3-bed7ff0767e2 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:38.845615 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e99be0-f224-4911-9cb3-bed7ff0767e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:38.845877 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e99be0-f224-4911-9cb3-bed7ff0767e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:38.847115 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30d72e-c84f-4769-a0bd-1b54ce68fbdd None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:38.858559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e99be0-f224-4911-9cb3-bed7ff0767e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:15:38.863606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-18e99be0-f224-4911-9cb3-bed7ff0767e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 14:15:38.863606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:38.863606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:38.863606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:38.863606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:38.863606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:38.863606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:38.863606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:38.863606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:38.863606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:38.863606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:38.863606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:38.863606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:15:38.863606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:15:38.863606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:15:38.863606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:15:38.864584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:15:38.864584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:15:38.864584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:15:38.864584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:15:38.864584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:38.864584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:38.864584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:38.864584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:38.864584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:38.864584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:38.864584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:38.864584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:38.864584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:38.864584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:38.864584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:38.864584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:38.865549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:38.865549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:38.865549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:38.865549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:38.865549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:15:38.865549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:15:38.865549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:15:38.865549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:15:38.865549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:15:38.865549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:15:38.865549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 14:15:38.865549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:38.865549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 519/1050] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Sep 5 14:15:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:38.871750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e99be0-f224-4911-9cb3-bed7ff0767e2 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:38.931593 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:38.940294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:38.967493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30d72e-c84f-4769-a0bd-1b54ce68fbdd None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:38.992369 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e99be0-f224-4911-9cb3-bed7ff0767e2 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:39.005493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32e635-6f0a-462c-9972-773266e0faab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:39.006130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32e635-6f0a-462c-9972-773266e0faab None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:39.006408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32e635-6f0a-462c-9972-773266e0faab None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:39.006676 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32e635-6f0a-462c-9972-773266e0faab None admin] PATH_INFO: `/v3/services/image` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:39.007760 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32e635-6f0a-462c-9972-773266e0faab None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:39.009357 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32e635-6f0a-462c-9972-773266e0faab None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:39.018390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-1d32e635-6f0a-462c-9972-773266e0faab None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 14:15:39.018390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:39.018390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:15:39.018390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:15:39.018390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:15:39.018390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:15:39.018390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:15:39.018390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:39.018390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 14:15:39.018390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:39.018390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:15:39.018390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:39.018390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:39.018390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:39.018390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:39.018390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:39.018390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:39.019519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:39.019519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:39.019519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:39.019519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:39.019519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:39.019519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:39.019519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:15:39.019519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:15:39.019519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:15:39.019519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:15:39.019519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:39.019519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:39.019519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:39.019519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:39.019519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:39.019519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:39.023541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:39.023541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:39.023541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:39.023541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:39.023541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:39.023541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:39.023541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:39.023541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:39.023541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:39.023541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:39.023541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:15:39.023541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:15:39.023541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 14:15:39.023541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:39.023541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 532/1051] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Sep 5 14:15:38 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:39.028823 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32e635-6f0a-462c-9972-773266e0faab None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:39.030716 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba620c-c902-4f6f-8f20-574e476bd0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:39.031367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba620c-c902-4f6f-8f20-574e476bd0ba None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:39.031650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba620c-c902-4f6f-8f20-574e476bd0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:39.031922 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba620c-c902-4f6f-8f20-574e476bd0ba None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:39.036110 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba620c-c902-4f6f-8f20-574e476bd0ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:39.036110 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba620c-c902-4f6f-8f20-574e476bd0ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:39.036110 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba620c-c902-4f6f-8f20-574e476bd0ba None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:39.045694 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 520/1052] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Sep 5 14:15:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:39.051452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba620c-c902-4f6f-8f20-574e476bd0ba None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:39.139769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32e635-6f0a-462c-9972-773266e0faab None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:39.165897 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba620c-c902-4f6f-8f20-574e476bd0ba None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:39.176219 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:39.179695 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6950f2-98b1-4fcf-9631-f5f5094abb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:39.180895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6950f2-98b1-4fcf-9631-f5f5094abb60 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:39.181179 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6950f2-98b1-4fcf-9631-f5f5094abb60 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:39.181448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6950f2-98b1-4fcf-9631-f5f5094abb60 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:39.182077 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6950f2-98b1-4fcf-9631-f5f5094abb60 None admin] RBAC: Adding query filter params (name=image) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:39.182380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6950f2-98b1-4fcf-9631-f5f5094abb60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:39.183824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6950f2-98b1-4fcf-9631-f5f5094abb60 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:39.194139 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 533/1053] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:39.194777 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6950f2-98b1-4fcf-9631-f5f5094abb60 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:39.198344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:39.200829 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ef9c7-52d2-43c8-8524-aac11f9f7dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:39.201460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ef9c7-52d2-43c8-8524-aac11f9f7dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:39.201737 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ef9c7-52d2-43c8-8524-aac11f9f7dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:39.202004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ef9c7-52d2-43c8-8524-aac11f9f7dc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:39.213201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ef9c7-52d2-43c8-8524-aac11f9f7dc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:39.214762 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ef9c7-52d2-43c8-8524-aac11f9f7dc0 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:39.230594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-a21ef9c7-52d2-43c8-8524-aac11f9f7dc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:39.230594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:39.230594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:39.230594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:39.230594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:39.230594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:39.230594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:39.230594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:39.230594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:39.230594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:39.230594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:39.230594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:39.230594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:39.230594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:39.230594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:39.230594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:39.231735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:39.231735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:39.231735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:39.231735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:39.231735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:39.231735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:39.231735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:39.231735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:39.231735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:39.231735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:39.231735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:39.231735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:39.231735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:39.231735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:39.231735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:39.231735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:39.231735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:39.232911 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:39.232911 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:39.232911 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:39.236433 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 521/1054] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:39.237294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ef9c7-52d2-43c8-8524-aac11f9f7dc0 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:39.304421 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6950f2-98b1-4fcf-9631-f5f5094abb60 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:39.342241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccec36-0ac8-49a7-b7e4-df0d8aedfd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:39.342977 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccec36-0ac8-49a7-b7e4-df0d8aedfd00 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:39.343257 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccec36-0ac8-49a7-b7e4-df0d8aedfd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:39.343527 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccec36-0ac8-49a7-b7e4-df0d8aedfd00 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:39.344147 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccec36-0ac8-49a7-b7e4-df0d8aedfd00 None admin] RBAC: Adding query filter params (type=image) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:39.344445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccec36-0ac8-49a7-b7e4-df0d8aedfd00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:39.345452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ef9c7-52d2-43c8-8524-aac11f9f7dc0 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:39.345803 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccec36-0ac8-49a7-b7e4-df0d8aedfd00 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:39.355487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 534/1055] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:39.356734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccec36-0ac8-49a7-b7e4-df0d8aedfd00 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:39.381125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac0814-fe40-469b-9bdc-9ef3aa063b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:39.381867 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac0814-fe40-469b-9bdc-9ef3aa063b6f None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:39.382158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac0814-fe40-469b-9bdc-9ef3aa063b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:39.382600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac0814-fe40-469b-9bdc-9ef3aa063b6f None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:39.383497 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac0814-fe40-469b-9bdc-9ef3aa063b6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:39.383844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac0814-fe40-469b-9bdc-9ef3aa063b6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:39.385381 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac0814-fe40-469b-9bdc-9ef3aa063b6f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:39.403183 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 522/1056] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:39.404186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac0814-fe40-469b-9bdc-9ef3aa063b6f None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:39.428290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:39.431968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:39.462538 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccec36-0ac8-49a7-b7e4-df0d8aedfd00 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:39.497708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a9f94-f9ae-4128-8caa-f498f84d75e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:39.498571 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a9f94-f9ae-4128-8caa-f498f84d75e1 None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:39.499035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a9f94-f9ae-4128-8caa-f498f84d75e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:39.499471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a9f94-f9ae-4128-8caa-f498f84d75e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:39.500131 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a9f94-f9ae-4128-8caa-f498f84d75e1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:39.501492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a9f94-f9ae-4128-8caa-f498f84d75e1 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:39.511114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac0814-fe40-469b-9bdc-9ef3aa063b6f None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:39.545791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31376c3c-5f33-425a-99a0-8e22117fbb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:39.546417 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31376c3c-5f33-425a-99a0-8e22117fbb33 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:39.546701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31376c3c-5f33-425a-99a0-8e22117fbb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:39.547030 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31376c3c-5f33-425a-99a0-8e22117fbb33 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:39.553896 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31376c3c-5f33-425a-99a0-8e22117fbb33 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:39.555426 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31376c3c-5f33-425a-99a0-8e22117fbb33 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:39.566886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-31376c3c-5f33-425a-99a0-8e22117fbb33 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:15:39.566886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:39.566886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:39.566886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:39.566886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:39.566886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:39.566886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:39.566886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:39.566886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:39.566886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:39.566886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:39.566886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:39.566886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:15:39.566886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:15:39.566886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:15:39.566886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:39.568088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:39.569205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:39.569205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:39.569205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:15:39.569205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:15:39.569205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:15:39.569205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:15:39.569205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:15:39.569205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:15:39.569205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:15:39.569205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:39.569205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 523/1057] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 5 14:15:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:39.572194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 535/1058] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:15:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:15:39.575807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31376c3c-5f33-425a-99a0-8e22117fbb33 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:39.667076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:39.686733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31376c3c-5f33-425a-99a0-8e22117fbb33 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:39.730278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e8ba2-5b8e-4cf9-84cd-12073c48ac62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:39.730918 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e8ba2-5b8e-4cf9-84cd-12073c48ac62 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:39.731199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e8ba2-5b8e-4cf9-84cd-12073c48ac62 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:39.731469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e8ba2-5b8e-4cf9-84cd-12073c48ac62 None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:39.732186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e8ba2-5b8e-4cf9-84cd-12073c48ac62 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:39.732488 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e8ba2-5b8e-4cf9-84cd-12073c48ac62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:39.733899 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e8ba2-5b8e-4cf9-84cd-12073c48ac62 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:39.750886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 524/1059] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 5 14:15:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:39.756695 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a9f94-f9ae-4128-8caa-f498f84d75e1 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:39.863409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a9f94-f9ae-4128-8caa-f498f84d75e1 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:39.886912 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:39.897991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7da6b3-0977-4b8c-bfe3-51fc536dffb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:39.899287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da6b3-0977-4b8c-bfe3-51fc536dffb7 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:39.899577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da6b3-0977-4b8c-bfe3-51fc536dffb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:39.899852 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da6b3-0977-4b8c-bfe3-51fc536dffb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:39.911646 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7da6b3-0977-4b8c-bfe3-51fc536dffb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:39.913162 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7da6b3-0977-4b8c-bfe3-51fc536dffb7 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:39.932714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-3f7da6b3-0977-4b8c-bfe3-51fc536dffb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:39.932714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:39.932714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:39.932714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:39.932714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:39.932714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:39.932714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:39.932714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:39.932714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:39.932714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:39.932714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:39.932714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:39.932714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:39.932714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:39.932714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:39.932714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:39.933700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:39.933700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:39.933700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:39.933700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:39.933700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:39.933700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:39.933700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:39.933700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:39.933700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:39.933700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:39.933700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:39.933700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:39.933700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:39.933700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:39.933700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:39.933700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:39.933700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:39.934617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:39.934617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:39.934617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:39.934617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 536/1060] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:39.936644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e8ba2-5b8e-4cf9-84cd-12073c48ac62 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:40.039283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e8ba2-5b8e-4cf9-84cd-12073c48ac62 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:40.076591 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc3ab8-782f-4018-ad7b-a3ebafed17a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:40.077199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc3ab8-782f-4018-ad7b-a3ebafed17a6 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:40.077481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc3ab8-782f-4018-ad7b-a3ebafed17a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:40.077762 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc3ab8-782f-4018-ad7b-a3ebafed17a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:40.078400 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc3ab8-782f-4018-ad7b-a3ebafed17a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:40.078702 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc3ab8-782f-4018-ad7b-a3ebafed17a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:40.080162 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc3ab8-782f-4018-ad7b-a3ebafed17a6 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:40.083134 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:40.099703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 525/1061] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:40.100968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7da6b3-0977-4b8c-bfe3-51fc536dffb7 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:40.203175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7da6b3-0977-4b8c-bfe3-51fc536dffb7 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:40.236454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100c67c-e925-4066-83c9-6fd7190b3039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:40.237091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100c67c-e925-4066-83c9-6fd7190b3039 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:40.237372 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100c67c-e925-4066-83c9-6fd7190b3039 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:40.237646 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100c67c-e925-4066-83c9-6fd7190b3039 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:40.244884 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100c67c-e925-4066-83c9-6fd7190b3039 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:40.246326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100c67c-e925-4066-83c9-6fd7190b3039 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:40.257435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-4100c67c-e925-4066-83c9-6fd7190b3039 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:40.257435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:40.257435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:40.257435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:40.257435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:40.257435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:40.257435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:40.257435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:40.257435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:40.257435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:40.257435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:40.257435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:40.257435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:15:40.257435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:15:40.257435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:15:40.258437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:15:40.258437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:40.258437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:40.258437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:40.258437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:40.258437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:40.258437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:40.258437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:40.258437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:40.258437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:40.258437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:40.258437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:40.258437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:40.258437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:40.258437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:40.258437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:40.258437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:40.259922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:15:40.259922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:15:40.259922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:15:40.259922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:15:40.259922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:15:40.259922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:15:40.259922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:40.259922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:40.260642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 537/1062] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:40.264352 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc3ab8-782f-4018-ad7b-a3ebafed17a6 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:40.369534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc3ab8-782f-4018-ad7b-a3ebafed17a6 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:40.403481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f4216-4a73-42ff-87ed-93b6ae9ed7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:40.404209 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f4216-4a73-42ff-87ed-93b6ae9ed7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:40.404634 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f4216-4a73-42ff-87ed-93b6ae9ed7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:40.405024 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f4216-4a73-42ff-87ed-93b6ae9ed7d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:40.405805 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f4216-4a73-42ff-87ed-93b6ae9ed7d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:40.406209 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f4216-4a73-42ff-87ed-93b6ae9ed7d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:40.407751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f4216-4a73-42ff-87ed-93b6ae9ed7d4 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:40.424777 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 526/1063] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:15:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:40.425943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100c67c-e925-4066-83c9-6fd7190b3039 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:40.464773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:40.532241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100c67c-e925-4066-83c9-6fd7190b3039 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:40.567427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4609a35-0e1f-4072-accb-46c04c1ba407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:40.568255 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4609a35-0e1f-4072-accb-46c04c1ba407 None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:40.568634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4609a35-0e1f-4072-accb-46c04c1ba407 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:40.569007 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4609a35-0e1f-4072-accb-46c04c1ba407 None admin] PATH_INFO: `/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/709b9ae82dcd47408fc9a6d54a54428a/roles/48a9bca3f43c4943855875bf97ec564f` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:40.571884 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4609a35-0e1f-4072-accb-46c04c1ba407 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9ca42deaed344eea726fe6e32356efc, user_id=709b9ae82dcd47408fc9a6d54a54428a, role_id=48a9bca3f43c4943855875bf97ec564f)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:40.573465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4609a35-0e1f-4072-accb-46c04c1ba407 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:40.650830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 538/1064] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:15:40 2022] PUT /identity/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/709b9ae82dcd47408fc9a6d54a54428a/roles/48a9bca3f43c4943855875bf97ec564f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:15:40.783415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177a62e-c737-4d0a-aea6-5beea2a151d2 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:40.783415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177a62e-c737-4d0a-aea6-5beea2a151d2 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:40.783415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177a62e-c737-4d0a-aea6-5beea2a151d2 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:40.784685 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 527/1065] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:40.790105 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbd245-c36b-4103-bfe1-3600f0e9cc4b None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:40.790648 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbd245-c36b-4103-bfe1-3600f0e9cc4b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:40.791175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbd245-c36b-4103-bfe1-3600f0e9cc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:40.812143 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-49cbd245-c36b-4103-bfe1-3600f0e9cc4b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:40.813359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-49cbd245-c36b-4103-bfe1-3600f0e9cc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:40.882586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 539/1066] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:41.152311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177a62e-c737-4d0a-aea6-5beea2a151d2 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:41.213238 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:41.257508 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177a62e-c737-4d0a-aea6-5beea2a151d2 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:41.292521 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce92613-58a9-4bdb-b099-54350561312c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:41.293151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce92613-58a9-4bdb-b099-54350561312c None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:41.293432 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce92613-58a9-4bdb-b099-54350561312c None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:41.293704 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce92613-58a9-4bdb-b099-54350561312c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:41.303815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce92613-58a9-4bdb-b099-54350561312c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:15:41.308587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-3ce92613-58a9-4bdb-b099-54350561312c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:15:41.308587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:41.308587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:41.308587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:41.308587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:41.308587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:41.308587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:41.308587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:41.308587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:41.308587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:41.308587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:41.308587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:41.308587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:15:41.308587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:15:41.308587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:15:41.308587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:15:41.309503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:15:41.309503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:15:41.309503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:15:41.309503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:15:41.309503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:41.309503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:41.309503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:41.309503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:41.309503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:41.309503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:41.309503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:41.309503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:41.309503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:41.309503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:41.309503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:41.309503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:41.310349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:41.310349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:41.310349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:41.310349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:41.310349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:15:41.310349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:15:41.310349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:15:41.310349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:15:41.310349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:15:41.310349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:15:41.310349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:15:41.310349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:41.311782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 528/1067] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:15:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:41.315690 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbd245-c36b-4103-bfe1-3600f0e9cc4b None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:41.344430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:41.415345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbd245-c36b-4103-bfe1-3600f0e9cc4b None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:41.449792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a174b-25f2-4280-9d58-4189871c2cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:41.450392 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a174b-25f2-4280-9d58-4189871c2cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:41.450670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a174b-25f2-4280-9d58-4189871c2cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:41.450972 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a174b-25f2-4280-9d58-4189871c2cc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:41.451627 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a174b-25f2-4280-9d58-4189871c2cc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:41.451926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a174b-25f2-4280-9d58-4189871c2cc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:41.453275 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a174b-25f2-4280-9d58-4189871c2cc5 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:41.466497 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 540/1068] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:15:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:41.467791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce92613-58a9-4bdb-b099-54350561312c None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:41.518838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca8a6c-4084-49fd-bac5-ee76d97274eb None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:41.519171 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca8a6c-4084-49fd-bac5-ee76d97274eb None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:41.519419 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca8a6c-4084-49fd-bac5-ee76d97274eb None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:41.520748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 541/1069] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:15:41.526364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4702e-9310-458a-b66c-92c5d3c5f7e7 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:41.526655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4702e-9310-458a-b66c-92c5d3c5f7e7 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:41.526963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4702e-9310-458a-b66c-92c5d3c5f7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:41.544739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-edb4702e-9310-458a-b66c-92c5d3c5f7e7 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:41.545921 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-edb4702e-9310-458a-b66c-92c5d3c5f7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:41.577196 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce92613-58a9-4bdb-b099-54350561312c None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:41.594834 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:41.614522 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2003f-a58e-4954-8118-af8a2ef0d029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:41.615228 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2003f-a58e-4954-8118-af8a2ef0d029 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:41.615496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2003f-a58e-4954-8118-af8a2ef0d029 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:41.615773 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2003f-a58e-4954-8118-af8a2ef0d029 None admin] PATH_INFO: `/v3/users/7056d65edb5945b8ba316ea7e705afea` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:41.626960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 542/1070] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:41.631537 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2003f-a58e-4954-8118-af8a2ef0d029 None admin] RBAC: Authorizing `identity:get_user(user_id=7056d65edb5945b8ba316ea7e705afea)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:41.633602 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2003f-a58e-4954-8118-af8a2ef0d029 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:41.641511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4702e-9310-458a-b66c-92c5d3c5f7e7 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:41.642225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 529/1071] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:15:41 2022] GET /identity/v3/users/7056d65edb5945b8ba316ea7e705afea => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:41.660618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:41.743393 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4702e-9310-458a-b66c-92c5d3c5f7e7 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:41.777091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e442c-353f-4f4b-becd-71e211ce52ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:41.777702 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e442c-353f-4f4b-becd-71e211ce52ba None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:41.777984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e442c-353f-4f4b-becd-71e211ce52ba None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:41.778263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e442c-353f-4f4b-becd-71e211ce52ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:41.844926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e442c-353f-4f4b-becd-71e211ce52ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:41.846373 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e442c-353f-4f4b-becd-71e211ce52ba None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:41.863897 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:41.896606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2003f-a58e-4954-8118-af8a2ef0d029 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:41.951518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 543/1072] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:15:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 14:15:41.956934 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e442c-353f-4f4b-becd-71e211ce52ba None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:42.003336 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2003f-a58e-4954-8118-af8a2ef0d029 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:42.037419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a71469-32d0-42f5-ab04-4059f156b0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:42.038031 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a71469-32d0-42f5-ab04-4059f156b0af None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:42.038309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a71469-32d0-42f5-ab04-4059f156b0af None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:42.038582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a71469-32d0-42f5-ab04-4059f156b0af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:42.061359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e442c-353f-4f4b-becd-71e211ce52ba None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:42.082898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:42.087633 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:42.105234 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca171e-678c-42e4-a65d-1149985bc695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:42.105887 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca171e-678c-42e4-a65d-1149985bc695 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:42.106175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca171e-678c-42e4-a65d-1149985bc695 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:42.106442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca171e-678c-42e4-a65d-1149985bc695 None admin] PATH_INFO: `/v3/projects/24c8d0395f524641bfe29e37e4b95ba6` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:42.107572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca171e-678c-42e4-a65d-1149985bc695 None admin] RBAC: Authorizing `identity:get_project(project_id=24c8d0395f524641bfe29e37e4b95ba6)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:42.108994 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca171e-678c-42e4-a65d-1149985bc695 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:42.110843 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 544/1073] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:15:41 2022] GET /identity/v3/projects/24c8d0395f524641bfe29e37e4b95ba6 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:42.116509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca171e-678c-42e4-a65d-1149985bc695 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:42.118763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a71469-32d0-42f5-ab04-4059f156b0af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:42.121458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a71469-32d0-42f5-ab04-4059f156b0af None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:42.225877 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 530/1074] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:15:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 14:15:42.232156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a71469-32d0-42f5-ab04-4059f156b0af None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:42.234493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca171e-678c-42e4-a65d-1149985bc695 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:42.278837 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f088a2-3e95-42ae-bc7b-a8f37ffc5b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:42.279383 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f088a2-3e95-42ae-bc7b-a8f37ffc5b8b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:42.279814 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f088a2-3e95-42ae-bc7b-a8f37ffc5b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:42.280096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f088a2-3e95-42ae-bc7b-a8f37ffc5b8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:42.281114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f088a2-3e95-42ae-bc7b-a8f37ffc5b8b None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=24c8d0395f524641bfe29e37e4b95ba6, user.id=7056d65edb5945b8ba316ea7e705afea) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:42.281426 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f088a2-3e95-42ae-bc7b-a8f37ffc5b8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:42.282865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f088a2-3e95-42ae-bc7b-a8f37ffc5b8b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:42.290642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 545/1075] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:15:42 2022] GET /identity/v3/role_assignments?user.id=7056d65edb5945b8ba316ea7e705afea&scope.project.id=24c8d0395f524641bfe29e37e4b95ba6&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:42.307961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:42.336893 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a71469-32d0-42f5-ab04-4059f156b0af None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:42.375961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fba97-7d2b-4ee6-805e-48e33d2ab17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:42.376610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fba97-7d2b-4ee6-805e-48e33d2ab17a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:42.377036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fba97-7d2b-4ee6-805e-48e33d2ab17a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:42.377205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fba97-7d2b-4ee6-805e-48e33d2ab17a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:42.389564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fba97-7d2b-4ee6-805e-48e33d2ab17a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:42.391135 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fba97-7d2b-4ee6-805e-48e33d2ab17a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:42.407504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-3f6fba97-7d2b-4ee6-805e-48e33d2ab17a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:42.407504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:42.407504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:42.407504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:42.407504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:42.407504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:42.407504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:42.407504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:42.407504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:42.407504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:42.407504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:42.407504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:42.407504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:42.407504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:42.407504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:42.407504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:42.408535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:42.408535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:42.408535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:42.408535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:42.408535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:42.408535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:42.408535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:42.408535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:42.408535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:42.408535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:42.408535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:42.408535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:42.408535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:42.408535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:42.408535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:42.408535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:42.408535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:42.409992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:42.409992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:42.409992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:42.409992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 531/1076] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:42.414762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f088a2-3e95-42ae-bc7b-a8f37ffc5b8b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:42.474092 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:42.524878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f088a2-3e95-42ae-bc7b-a8f37ffc5b8b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:42.548447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a688a0d-511f-4c4f-87ef-1972da3256b3 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:42.548658 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a688a0d-511f-4c4f-87ef-1972da3256b3 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:42.548937 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a688a0d-511f-4c4f-87ef-1972da3256b3 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:42.550264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 532/1077] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:15:42.555697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a30ec1-9756-4423-8805-25364167f2bd None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:42.556023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a30ec1-9756-4423-8805-25364167f2bd None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:42.556301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a30ec1-9756-4423-8805-25364167f2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:42.560177 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca91305-a120-4e94-b3b1-f5e905cc4f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:42.561010 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca91305-a120-4e94-b3b1-f5e905cc4f0a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:42.561440 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca91305-a120-4e94-b3b1-f5e905cc4f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:42.561880 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca91305-a120-4e94-b3b1-f5e905cc4f0a None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:42.562686 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca91305-a120-4e94-b3b1-f5e905cc4f0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:42.563182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca91305-a120-4e94-b3b1-f5e905cc4f0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:42.564772 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca91305-a120-4e94-b3b1-f5e905cc4f0a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:42.575809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-91a30ec1-9756-4423-8805-25364167f2bd None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:42.577474 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-91a30ec1-9756-4423-8805-25364167f2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:42.578957 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 546/1078] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:42.651721 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 533/1079] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:42.921696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca91305-a120-4e94-b3b1-f5e905cc4f0a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:43.018223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca91305-a120-4e94-b3b1-f5e905cc4f0a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:43.029141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:43.051732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9b25a-57a0-4cc4-af18-5ce765af83b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:43.052358 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9b25a-57a0-4cc4-af18-5ce765af83b0 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:43.052641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9b25a-57a0-4cc4-af18-5ce765af83b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:43.052914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9b25a-57a0-4cc4-af18-5ce765af83b0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:43.063989 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f9b25a-57a0-4cc4-af18-5ce765af83b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:15:43.069159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-22f9b25a-57a0-4cc4-af18-5ce765af83b0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 14:15:43.069159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:43.069159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:43.069159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:43.069159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:43.069159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:43.069159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:43.069159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:43.069159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:43.069159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:43.069159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:43.069159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:43.069159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:15:43.069159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:15:43.069159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:15:43.069159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:15:43.070324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:15:43.070324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:15:43.070324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:15:43.070324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:15:43.070324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:43.070324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:43.070324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:43.070324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:43.070324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:43.070324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:43.070324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:43.070324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:43.070324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:43.070324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:43.070324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:43.070324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:43.071987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:43.071987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:43.071987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:43.071987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:43.071987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:15:43.071987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:15:43.071987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:15:43.071987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:15:43.071987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:15:43.071987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:15:43.071987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 14:15:43.071987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:43.071987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 547/1080] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Sep 5 14:15:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:43.077008 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a30ec1-9756-4423-8805-25364167f2bd None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:43.131722 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:43.175136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a30ec1-9756-4423-8805-25364167f2bd None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:43.208533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41cfdc-7d22-4a6a-a1c9-6d466df53a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:43.209134 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41cfdc-7d22-4a6a-a1c9-6d466df53a5e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:43.209415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41cfdc-7d22-4a6a-a1c9-6d466df53a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:43.209686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41cfdc-7d22-4a6a-a1c9-6d466df53a5e None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:43.210345 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41cfdc-7d22-4a6a-a1c9-6d466df53a5e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:43.210647 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41cfdc-7d22-4a6a-a1c9-6d466df53a5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:43.212062 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41cfdc-7d22-4a6a-a1c9-6d466df53a5e None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:43.219674 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 534/1081] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Sep 5 14:15:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:43.225473 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9b25a-57a0-4cc4-af18-5ce765af83b0 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:43.327019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9b25a-57a0-4cc4-af18-5ce765af83b0 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:43.360372 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c67b4e-a7a0-4c12-9825-b2894b56683a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:43.361021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c67b4e-a7a0-4c12-9825-b2894b56683a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:43.361303 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c67b4e-a7a0-4c12-9825-b2894b56683a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:43.361580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c67b4e-a7a0-4c12-9825-b2894b56683a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:43.374001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c67b4e-a7a0-4c12-9825-b2894b56683a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:43.375533 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c67b4e-a7a0-4c12-9825-b2894b56683a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:43.390743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-43c67b4e-a7a0-4c12-9825-b2894b56683a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:43.390743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:43.390743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:43.390743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:43.390743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:43.390743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:43.390743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:43.390743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:43.390743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:43.390743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:43.390743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:43.390743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:43.390743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:43.390743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:43.390743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:43.390743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:43.391711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:43.391711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:43.391711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:43.391711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:43.391711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:43.391711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:43.391711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:43.391711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:43.391711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:43.391711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:43.391711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:43.391711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:43.391711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:43.391711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:43.391711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:43.391711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:43.391711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:43.392634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:43.392634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:43.392634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:43.392634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 548/1082] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:43.397772 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41cfdc-7d22-4a6a-a1c9-6d466df53a5e None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:43.493921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41cfdc-7d22-4a6a-a1c9-6d466df53a5e None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:43.510378 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:43.526492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e0edc-1083-4cc9-9bbf-4f2bb1fb8f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:43.527125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e0edc-1083-4cc9-9bbf-4f2bb1fb8f7d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:43.527409 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e0edc-1083-4cc9-9bbf-4f2bb1fb8f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:43.527681 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e0edc-1083-4cc9-9bbf-4f2bb1fb8f7d None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:43.528313 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e0edc-1083-4cc9-9bbf-4f2bb1fb8f7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:43.528617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e0edc-1083-4cc9-9bbf-4f2bb1fb8f7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:43.529975 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e0edc-1083-4cc9-9bbf-4f2bb1fb8f7d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:43.542394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 535/1083] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:43.548001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c67b4e-a7a0-4c12-9825-b2894b56683a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:43.573025 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:43.646398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c67b4e-a7a0-4c12-9825-b2894b56683a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:43.678810 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd51042-d6d4-4090-8a15-75a74285dc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:43.679464 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd51042-d6d4-4090-8a15-75a74285dc62 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:43.679747 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd51042-d6d4-4090-8a15-75a74285dc62 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:43.680593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd51042-d6d4-4090-8a15-75a74285dc62 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:43.688034 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd51042-d6d4-4090-8a15-75a74285dc62 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:43.689464 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd51042-d6d4-4090-8a15-75a74285dc62 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:43.700509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-cbd51042-d6d4-4090-8a15-75a74285dc62 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:15:43.700509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:43.700509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:43.700509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:43.700509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:43.700509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:43.700509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:43.700509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:43.700509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:43.700509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:43.700509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:43.700509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:43.700509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:15:43.700509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:15:43.700509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:15:43.700509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:43.706350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:43.707422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:43.707422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:43.707422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:15:43.707422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:15:43.707422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:15:43.707422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:15:43.707422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:15:43.707422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:15:43.707422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 14:15:43.707422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:43.707422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 549/1084] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 5 14:15:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:43.708030 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e0edc-1083-4cc9-9bbf-4f2bb1fb8f7d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:43.804829 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e0edc-1083-4cc9-9bbf-4f2bb1fb8f7d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:43.837803 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ded76-21d3-457e-afb5-3134c2cb4187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:43.838405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ded76-21d3-457e-afb5-3134c2cb4187 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:43.838685 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ded76-21d3-457e-afb5-3134c2cb4187 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:43.838985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ded76-21d3-457e-afb5-3134c2cb4187 None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:43.839705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ded76-21d3-457e-afb5-3134c2cb4187 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:43.840010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ded76-21d3-457e-afb5-3134c2cb4187 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:43.841419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ded76-21d3-457e-afb5-3134c2cb4187 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:43.859250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 536/1085] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 5 14:15:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:43.864983 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd51042-d6d4-4090-8a15-75a74285dc62 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:43.945014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:43.962707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd51042-d6d4-4090-8a15-75a74285dc62 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:43.995325 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b06718-644b-48ee-82bc-aaeb1c2da595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:43.995930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b06718-644b-48ee-82bc-aaeb1c2da595 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:43.996210 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b06718-644b-48ee-82bc-aaeb1c2da595 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:43.996484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b06718-644b-48ee-82bc-aaeb1c2da595 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:44.060958 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b06718-644b-48ee-82bc-aaeb1c2da595 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:44.062315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b06718-644b-48ee-82bc-aaeb1c2da595 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:44.141680 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:44.159391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 550/1086] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:15:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 14:15:44.165798 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ded76-21d3-457e-afb5-3134c2cb4187 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:44.200391 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:44.229703 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967eb6e-1644-421d-a14a-2ba955d6f8c2 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:44.229987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967eb6e-1644-421d-a14a-2ba955d6f8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:44.230261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967eb6e-1644-421d-a14a-2ba955d6f8c2 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:44.231629 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 551/1087] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:15:44.236564 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758030a8-2247-4d6c-a8fd-3cdfa8f92352 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:44.236847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758030a8-2247-4d6c-a8fd-3cdfa8f92352 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:44.237123 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758030a8-2247-4d6c-a8fd-3cdfa8f92352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:44.255270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-758030a8-2247-4d6c-a8fd-3cdfa8f92352 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:44.256445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-758030a8-2247-4d6c-a8fd-3cdfa8f92352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:44.263334 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ded76-21d3-457e-afb5-3134c2cb4187 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:44.296712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d00fff-44bb-45a0-bf53-333ae3a67828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:44.297339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d00fff-44bb-45a0-bf53-333ae3a67828 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:44.297623 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d00fff-44bb-45a0-bf53-333ae3a67828 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:44.297901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d00fff-44bb-45a0-bf53-333ae3a67828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:44.307858 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d00fff-44bb-45a0-bf53-333ae3a67828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:44.309292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d00fff-44bb-45a0-bf53-333ae3a67828 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:44.322806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 552/1088] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:15:44.324085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-00d00fff-44bb-45a0-bf53-333ae3a67828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:44.324085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:44.324085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:44.324085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:44.324085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:44.324085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:44.324085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:44.324085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:44.324085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:44.324085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:44.324085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:44.324085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:44.324085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:15:44.324085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:15:44.324085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:15:44.324085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:15:44.325173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:44.325173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:44.325173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:44.325173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:44.325173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:44.325173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:44.325173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:44.325173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:44.325173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:44.325173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:44.325173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:44.325173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:44.325173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:44.325173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:44.325173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:44.325173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:44.325173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:15:44.326101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:15:44.326101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:15:44.326101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:44.326101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 537/1089] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:44.330756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758030a8-2247-4d6c-a8fd-3cdfa8f92352 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:44.427350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758030a8-2247-4d6c-a8fd-3cdfa8f92352 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:44.440499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:44.460298 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10721876-9507-4401-98c6-57a1c0299d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:44.460892 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10721876-9507-4401-98c6-57a1c0299d51 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:44.461197 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10721876-9507-4401-98c6-57a1c0299d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:44.461467 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10721876-9507-4401-98c6-57a1c0299d51 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:44.462156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10721876-9507-4401-98c6-57a1c0299d51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:44.462455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10721876-9507-4401-98c6-57a1c0299d51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:44.463858 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10721876-9507-4401-98c6-57a1c0299d51 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:44.474706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 553/1090] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:44.480967 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d00fff-44bb-45a0-bf53-333ae3a67828 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:44.568540 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:44.581654 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d00fff-44bb-45a0-bf53-333ae3a67828 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:44.593035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10721876-9507-4401-98c6-57a1c0299d51 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:44.614256 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25d3b2-9597-45af-bfb2-9e7b0a28194b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:44.614919 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25d3b2-9597-45af-bfb2-9e7b0a28194b None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:44.615200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25d3b2-9597-45af-bfb2-9e7b0a28194b None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:44.615471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25d3b2-9597-45af-bfb2-9e7b0a28194b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:44.622469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25d3b2-9597-45af-bfb2-9e7b0a28194b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:44.623933 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25d3b2-9597-45af-bfb2-9e7b0a28194b None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:44.635338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-3d25d3b2-9597-45af-bfb2-9e7b0a28194b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:44.635338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:44.635338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:44.635338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:44.635338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:44.635338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:44.635338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:44.635338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:44.635338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:44.635338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:44.635338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:44.635338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:44.635338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:15:44.635338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:15:44.635338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:15:44.636371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:15:44.636371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:44.636371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:44.636371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:44.636371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:44.636371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:44.636371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:44.636371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:44.636371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:44.636371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:44.636371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:44.636371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:44.636371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:44.636371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:44.636371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:44.636371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:44.636371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:44.642273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:15:44.642273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:15:44.642273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:15:44.642273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:15:44.642273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:15:44.642273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:15:44.642273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:15:44.642273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:44.642273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 538/1091] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:15:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:44.642917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25d3b2-9597-45af-bfb2-9e7b0a28194b None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:44.695021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10721876-9507-4401-98c6-57a1c0299d51 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:44.727248 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575088f5-6557-413e-891e-59c7bf516fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:44.727867 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575088f5-6557-413e-891e-59c7bf516fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:44.728144 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575088f5-6557-413e-891e-59c7bf516fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:44.728410 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575088f5-6557-413e-891e-59c7bf516fc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:44.737978 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575088f5-6557-413e-891e-59c7bf516fc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:15:44.739598 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25d3b2-9597-45af-bfb2-9e7b0a28194b None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:44.743595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-575088f5-6557-413e-891e-59c7bf516fc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:15:44.743595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:44.743595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:44.743595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:44.743595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:44.743595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:44.743595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:44.743595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:44.743595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:44.743595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:44.743595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:44.743595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:44.743595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:15:44.743595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:15:44.743595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:15:44.743595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:15:44.744741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:15:44.744741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:15:44.744741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:15:44.744741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:15:44.744741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:44.744741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:44.744741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:44.744741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:44.744741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:44.744741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:44.744741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:44.744741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:44.744741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:44.744741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:44.744741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:44.744741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:44.745691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:44.745691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:44.745691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:44.745691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:44.745691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:15:44.745691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:15:44.745691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:15:44.745691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:15:44.745691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:15:44.745691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:15:44.745691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:15:44.745691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:15:44.745691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 554/1092] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:15:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:15:44.750480 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575088f5-6557-413e-891e-59c7bf516fc6 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:44.773707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1146032-2980-4cd7-a3b3-491cf4ce8cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:44.774309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1146032-2980-4cd7-a3b3-491cf4ce8cbf None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:44.774591 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1146032-2980-4cd7-a3b3-491cf4ce8cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:44.774906 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1146032-2980-4cd7-a3b3-491cf4ce8cbf None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:44.775591 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1146032-2980-4cd7-a3b3-491cf4ce8cbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:44.775896 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1146032-2980-4cd7-a3b3-491cf4ce8cbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:44.777320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1146032-2980-4cd7-a3b3-491cf4ce8cbf None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:44.780768 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:44.789424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 539/1093] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:15:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:44.795230 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1146032-2980-4cd7-a3b3-491cf4ce8cbf None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:44.805733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:44.849846 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575088f5-6557-413e-891e-59c7bf516fc6 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:44.882798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3488c12-bca0-40e7-ad17-8dc3757329cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:44.883968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3488c12-bca0-40e7-ad17-8dc3757329cc None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:44.884246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3488c12-bca0-40e7-ad17-8dc3757329cc None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:44.884524 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3488c12-bca0-40e7-ad17-8dc3757329cc None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:44.885183 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3488c12-bca0-40e7-ad17-8dc3757329cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:44.885497 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3488c12-bca0-40e7-ad17-8dc3757329cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:44.886882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3488c12-bca0-40e7-ad17-8dc3757329cc None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:44.892727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1146032-2980-4cd7-a3b3-491cf4ce8cbf None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:44.895153 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 555/1094] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:15:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:44.900941 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3488c12-bca0-40e7-ad17-8dc3757329cc None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:44.927444 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c742648-a14f-46be-9615-7348faaddb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:44.928041 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c742648-a14f-46be-9615-7348faaddb59 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:44.928315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c742648-a14f-46be-9615-7348faaddb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:44.928590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c742648-a14f-46be-9615-7348faaddb59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:44.929297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c742648-a14f-46be-9615-7348faaddb59 None admin] RBAC: Adding query filter params (role.id=48a9bca3f43c4943855875bf97ec564f, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=709b9ae82dcd47408fc9a6d54a54428a) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:44.929606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c742648-a14f-46be-9615-7348faaddb59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:44.931131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c742648-a14f-46be-9615-7348faaddb59 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:44.936686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 540/1095] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:15:44 2022] GET /identity/v3/role_assignments?user.id=709b9ae82dcd47408fc9a6d54a54428a&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=48a9bca3f43c4943855875bf97ec564f => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:44.996578 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:45.003207 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3488c12-bca0-40e7-ad17-8dc3757329cc None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:45.039798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f265a3b-eef5-42b6-a354-86a0928375e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:45.040456 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f265a3b-eef5-42b6-a354-86a0928375e5 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:45.040740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f265a3b-eef5-42b6-a354-86a0928375e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:45.041017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f265a3b-eef5-42b6-a354-86a0928375e5 None admin] PATH_INFO: `/v3/users/7056d65edb5945b8ba316ea7e705afea` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:45.042216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f265a3b-eef5-42b6-a354-86a0928375e5 None admin] RBAC: Authorizing `identity:get_user(user_id=7056d65edb5945b8ba316ea7e705afea)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:45.043714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f265a3b-eef5-42b6-a354-86a0928375e5 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:45.045360 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 556/1096] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:15:44 2022] GET /identity/v3/users/7056d65edb5945b8ba316ea7e705afea => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:45.051298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c742648-a14f-46be-9615-7348faaddb59 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:45.127926 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:45.156676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c742648-a14f-46be-9615-7348faaddb59 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:45.192468 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4307b-5b11-4d40-a882-a2a33d746e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:45.193096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4307b-5b11-4d40-a882-a2a33d746e3d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:45.193372 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4307b-5b11-4d40-a882-a2a33d746e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:45.193642 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4307b-5b11-4d40-a882-a2a33d746e3d None admin] PATH_INFO: `/v3/projects/24c8d0395f524641bfe29e37e4b95ba6` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:45.194737 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4307b-5b11-4d40-a882-a2a33d746e3d None admin] RBAC: Authorizing `identity:get_project(project_id=24c8d0395f524641bfe29e37e4b95ba6)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:45.196730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4307b-5b11-4d40-a882-a2a33d746e3d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:45.198356 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 541/1097] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:15:45 2022] GET /identity/v3/projects/24c8d0395f524641bfe29e37e4b95ba6 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:45.204481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f265a3b-eef5-42b6-a354-86a0928375e5 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:45.311395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f265a3b-eef5-42b6-a354-86a0928375e5 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:45.345725 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121fcde-b786-495f-b7e5-9d72a43a7050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:45.346384 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121fcde-b786-495f-b7e5-9d72a43a7050 None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:45.346664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121fcde-b786-495f-b7e5-9d72a43a7050 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:45.346971 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121fcde-b786-495f-b7e5-9d72a43a7050 None admin] PATH_INFO: `/v3/projects/24c8d0395f524641bfe29e37e4b95ba6/users/7056d65edb5945b8ba316ea7e705afea/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:45.348891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121fcde-b786-495f-b7e5-9d72a43a7050 None admin] RBAC: Authorizing `identity:create_grant(project_id=24c8d0395f524641bfe29e37e4b95ba6, user_id=7056d65edb5945b8ba316ea7e705afea, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:45.350309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121fcde-b786-495f-b7e5-9d72a43a7050 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:45.438950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 557/1098] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:15:45 2022] PUT /identity/v3/projects/24c8d0395f524641bfe29e37e4b95ba6/users/7056d65edb5945b8ba316ea7e705afea/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:15:47.333901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6bbe9-bd29-4797-9418-427da1677fbd None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:47.334408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6bbe9-bd29-4797-9418-427da1677fbd None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:47.334521 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6bbe9-bd29-4797-9418-427da1677fbd None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:47.335829 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 542/1099] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:47.341582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269ec27-fbdb-4ea1-944d-52708462199f None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:47.342002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269ec27-fbdb-4ea1-944d-52708462199f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:47.342131 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269ec27-fbdb-4ea1-944d-52708462199f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:47.359797 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-2269ec27-fbdb-4ea1-944d-52708462199f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:47.360900 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-2269ec27-fbdb-4ea1-944d-52708462199f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:47.376565 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:47.424649 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 558/1100] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:47.690880 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6bbe9-bd29-4797-9418-427da1677fbd None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:47.790338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6bbe9-bd29-4797-9418-427da1677fbd None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:47.822825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930489fb-ae8a-40b1-82a8-64d5389c6181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:47.823446 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930489fb-ae8a-40b1-82a8-64d5389c6181 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:47.823727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930489fb-ae8a-40b1-82a8-64d5389c6181 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:47.824000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930489fb-ae8a-40b1-82a8-64d5389c6181 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:47.831322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:47.834383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930489fb-ae8a-40b1-82a8-64d5389c6181 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:15:47.839604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-930489fb-ae8a-40b1-82a8-64d5389c6181 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:15:47.839604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:15:47.839604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:15:47.839604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:15:47.839604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:15:47.839604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:15:47.839604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:15:47.839604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:15:47.839604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:15:47.839604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:15:47.839604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:15:47.839604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:15:47.839604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:15:47.839604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:15:47.839604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:15:47.839604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:15:47.840692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:15:47.840692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:15:47.840692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:15:47.840692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:15:47.840692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:15:47.840692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:15:47.840692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:15:47.840692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:15:47.840692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:15:47.840692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:15:47.840692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:15:47.840692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:15:47.840692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:15:47.840692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:15:47.840692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:15:47.840692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:15:47.841871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:15:47.841871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:15:47.841871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:15:47.841871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:15:47.841871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:15:47.841871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:15:47.841871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:15:47.841871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:15:47.841871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:15:47.841871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:15:47.841871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:15:47.841871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:15:47.841871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 543/1101] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:15:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:15:47.846504 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269ec27-fbdb-4ea1-944d-52708462199f None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:47.941337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269ec27-fbdb-4ea1-944d-52708462199f None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:47.974686 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8ea64-ab09-4443-b8d8-b59cbfb9d5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:47.975297 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8ea64-ab09-4443-b8d8-b59cbfb9d5d7 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:47.975576 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8ea64-ab09-4443-b8d8-b59cbfb9d5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:47.975844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8ea64-ab09-4443-b8d8-b59cbfb9d5d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:47.976491 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8ea64-ab09-4443-b8d8-b59cbfb9d5d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:47.977322 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8ea64-ab09-4443-b8d8-b59cbfb9d5d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:47.978666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8ea64-ab09-4443-b8d8-b59cbfb9d5d7 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:47.986316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 559/1102] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:15:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:47.991921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930489fb-ae8a-40b1-82a8-64d5389c6181 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:48.088875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930489fb-ae8a-40b1-82a8-64d5389c6181 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:48.122228 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1a3fd-2c50-42a8-a017-341cf8f7cb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:48.122875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1a3fd-2c50-42a8-a017-341cf8f7cb1d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:48.123158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1a3fd-2c50-42a8-a017-341cf8f7cb1d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:48.123432 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1a3fd-2c50-42a8-a017-341cf8f7cb1d None admin] PATH_INFO: `/v3/users/7056d65edb5945b8ba316ea7e705afea` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:48.124538 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1a3fd-2c50-42a8-a017-341cf8f7cb1d None admin] RBAC: Authorizing `identity:get_user(user_id=7056d65edb5945b8ba316ea7e705afea)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:48.125984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1a3fd-2c50-42a8-a017-341cf8f7cb1d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:48.127568 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 544/1103] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:15:47 2022] GET /identity/v3/users/7056d65edb5945b8ba316ea7e705afea => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:48.133648 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8ea64-ab09-4443-b8d8-b59cbfb9d5d7 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:48.149286 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:48.230374 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8ea64-ab09-4443-b8d8-b59cbfb9d5d7 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:48.262288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25d8d9-ab6d-45cd-8491-a66a5e336e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:48.262910 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25d8d9-ab6d-45cd-8491-a66a5e336e63 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:48.263191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25d8d9-ab6d-45cd-8491-a66a5e336e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:48.263459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25d8d9-ab6d-45cd-8491-a66a5e336e63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:48.328180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25d8d9-ab6d-45cd-8491-a66a5e336e63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:48.329604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25d8d9-ab6d-45cd-8491-a66a5e336e63 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:48.343345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:48.424538 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 560/1104] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:15:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:15:48.431061 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1a3fd-2c50-42a8-a017-341cf8f7cb1d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:48.494613 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:48.526558 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1a3fd-2c50-42a8-a017-341cf8f7cb1d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:48.558373 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196d2d6-0d7f-4a25-9f27-d448a766de50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:48.559006 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196d2d6-0d7f-4a25-9f27-d448a766de50 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:48.559288 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196d2d6-0d7f-4a25-9f27-d448a766de50 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:48.559562 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196d2d6-0d7f-4a25-9f27-d448a766de50 None admin] PATH_INFO: `/v3/projects/24c8d0395f524641bfe29e37e4b95ba6` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:48.560659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196d2d6-0d7f-4a25-9f27-d448a766de50 None admin] RBAC: Authorizing `identity:get_project(project_id=24c8d0395f524641bfe29e37e4b95ba6)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:48.562081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196d2d6-0d7f-4a25-9f27-d448a766de50 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:48.563697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 545/1105] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:15:48 2022] GET /identity/v3/projects/24c8d0395f524641bfe29e37e4b95ba6 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:48.569898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25d8d9-ab6d-45cd-8491-a66a5e336e63 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:48.665026 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25d8d9-ab6d-45cd-8491-a66a5e336e63 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:48.678249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:48.697989 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8502d-5b7a-4673-8f86-47034b6c6f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:48.698578 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8502d-5b7a-4673-8f86-47034b6c6f0b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:48.698876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8502d-5b7a-4673-8f86-47034b6c6f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:48.699149 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8502d-5b7a-4673-8f86-47034b6c6f0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:48.699856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8502d-5b7a-4673-8f86-47034b6c6f0b None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=24c8d0395f524641bfe29e37e4b95ba6, user.id=7056d65edb5945b8ba316ea7e705afea) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:48.700161 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8502d-5b7a-4673-8f86-47034b6c6f0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:48.701564 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8502d-5b7a-4673-8f86-47034b6c6f0b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:48.707181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 561/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:15:48 2022] GET /identity/v3/role_assignments?user.id=7056d65edb5945b8ba316ea7e705afea&scope.project.id=24c8d0395f524641bfe29e37e4b95ba6&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:50.607897 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76b73d-f51e-44a7-89df-468fcb2800c8 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:50.608428 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76b73d-f51e-44a7-89df-468fcb2800c8 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:50.608428 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76b73d-f51e-44a7-89df-468fcb2800c8 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:50.609779 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 546/1107] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:50.614987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb8f7b-8066-497a-92aa-7e7c4d102647 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:50.615423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb8f7b-8066-497a-92aa-7e7c4d102647 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:50.615519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb8f7b-8066-497a-92aa-7e7c4d102647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:50.634535 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-1beb8f7b-8066-497a-92aa-7e7c4d102647 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:50.635781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-1beb8f7b-8066-497a-92aa-7e7c4d102647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:50.701318 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 562/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:50.967948 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76b73d-f51e-44a7-89df-468fcb2800c8 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:51.066117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76b73d-f51e-44a7-89df-468fcb2800c8 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:51.099861 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648422b-bcde-4180-8bde-8b8144eff83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:51.100480 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648422b-bcde-4180-8bde-8b8144eff83f None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:51.100756 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648422b-bcde-4180-8bde-8b8144eff83f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:51.101023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648422b-bcde-4180-8bde-8b8144eff83f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:51.102135 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648422b-bcde-4180-8bde-8b8144eff83f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:51.103583 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648422b-bcde-4180-8bde-8b8144eff83f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:51.105111 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 547/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:15:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:51.110765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb8f7b-8066-497a-92aa-7e7c4d102647 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:51.206004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb8f7b-8066-497a-92aa-7e7c4d102647 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:51.215430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:51.238843 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266a906-1655-472f-a698-cd3c11e177cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:51.239395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266a906-1655-472f-a698-cd3c11e177cd None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:51.239679 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266a906-1655-472f-a698-cd3c11e177cd None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:51.239956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266a906-1655-472f-a698-cd3c11e177cd None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:51.240596 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266a906-1655-472f-a698-cd3c11e177cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:51.242015 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266a906-1655-472f-a698-cd3c11e177cd None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:51.322958 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 563/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:15:51 2022] POST /identity/v3/users => generated 286 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:15:53.253693 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdd8dd-3e64-4420-9b91-59c42756938d None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:53.254305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdd8dd-3e64-4420-9b91-59c42756938d None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:53.254305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdd8dd-3e64-4420-9b91-59c42756938d None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:53.255586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 548/1111] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:53.260775 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cda6b4-47b7-46b4-b982-a313912338ff None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:53.261221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cda6b4-47b7-46b4-b982-a313912338ff None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:53.261321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cda6b4-47b7-46b4-b982-a313912338ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:53.279718 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-01cda6b4-47b7-46b4-b982-a313912338ff None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:53.280931 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-01cda6b4-47b7-46b4-b982-a313912338ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:53.346065 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 564/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:53.614358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cdd8dd-3e64-4420-9b91-59c42756938d None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:53.618686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:53.713072 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cdd8dd-3e64-4420-9b91-59c42756938d None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:53.745458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55dd15f-ff8c-4fb9-9cfd-f71e55177338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:53.746074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dd15f-ff8c-4fb9-9cfd-f71e55177338 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:53.746356 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dd15f-ff8c-4fb9-9cfd-f71e55177338 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:53.746632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dd15f-ff8c-4fb9-9cfd-f71e55177338 None admin] PATH_INFO: `/v3/roles/1182ffc3e9f9477db2003500b7bfd402` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:53.748207 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55dd15f-ff8c-4fb9-9cfd-f71e55177338 None admin] RBAC: Authorizing `identity:get_role(role_id=1182ffc3e9f9477db2003500b7bfd402)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:53.749581 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55dd15f-ff8c-4fb9-9cfd-f71e55177338 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:53.751067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 549/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:15:53 2022] GET /identity/v3/roles/1182ffc3e9f9477db2003500b7bfd402 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:53.756457 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cda6b4-47b7-46b4-b982-a313912338ff None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:53.845029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:53.851379 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cda6b4-47b7-46b4-b982-a313912338ff None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:53.885635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916fb8f-4484-481b-a9e1-37d3e2b5116b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:53.886201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916fb8f-4484-481b-a9e1-37d3e2b5116b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:53.886483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916fb8f-4484-481b-a9e1-37d3e2b5116b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:53.886757 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916fb8f-4484-481b-a9e1-37d3e2b5116b None admin] PATH_INFO: `/v3/users/1a817d3534b04f63969a95d3bb500533` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:53.899570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916fb8f-4484-481b-a9e1-37d3e2b5116b None admin] RBAC: Authorizing `identity:get_user(user_id=1a817d3534b04f63969a95d3bb500533)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:53.901597 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916fb8f-4484-481b-a9e1-37d3e2b5116b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:53.903255 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 565/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:15:53 2022] GET /identity/v3/users/1a817d3534b04f63969a95d3bb500533 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:15:53.909153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55dd15f-ff8c-4fb9-9cfd-f71e55177338 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:53.979729 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:54.009122 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55dd15f-ff8c-4fb9-9cfd-f71e55177338 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:54.042325 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b073099-bcf1-4bac-8fd7-bb656e1d1a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:54.042954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b073099-bcf1-4bac-8fd7-bb656e1d1a5e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:54.043243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b073099-bcf1-4bac-8fd7-bb656e1d1a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:54.043520 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b073099-bcf1-4bac-8fd7-bb656e1d1a5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:54.108548 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b073099-bcf1-4bac-8fd7-bb656e1d1a5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:54.110070 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b073099-bcf1-4bac-8fd7-bb656e1d1a5e None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:54.174634 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:54.204150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 550/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:15:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:15:54.209821 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916fb8f-4484-481b-a9e1-37d3e2b5116b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:54.306109 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916fb8f-4484-481b-a9e1-37d3e2b5116b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:54.338797 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e3b73-15aa-4117-9b45-c0b9f6f61374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:54.339437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e3b73-15aa-4117-9b45-c0b9f6f61374 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:54.339717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e3b73-15aa-4117-9b45-c0b9f6f61374 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:54.339992 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e3b73-15aa-4117-9b45-c0b9f6f61374 None admin] PATH_INFO: `/v3/projects/24c8d0395f524641bfe29e37e4b95ba6` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:54.341091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e3b73-15aa-4117-9b45-c0b9f6f61374 None admin] RBAC: Authorizing `identity:get_project(project_id=24c8d0395f524641bfe29e37e4b95ba6)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:54.342504 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e3b73-15aa-4117-9b45-c0b9f6f61374 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:54.344111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 566/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:15:54 2022] GET /identity/v3/projects/24c8d0395f524641bfe29e37e4b95ba6 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:54.349903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b073099-bcf1-4bac-8fd7-bb656e1d1a5e None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:54.444863 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b073099-bcf1-4bac-8fd7-bb656e1d1a5e None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:54.477539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d88a3-bdd7-4ec0-9bcd-fba5e0bf36f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:54.478123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d88a3-bdd7-4ec0-9bcd-fba5e0bf36f2 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:54.478398 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d88a3-bdd7-4ec0-9bcd-fba5e0bf36f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:54.478666 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d88a3-bdd7-4ec0-9bcd-fba5e0bf36f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:54.479402 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d88a3-bdd7-4ec0-9bcd-fba5e0bf36f2 None admin] RBAC: Adding query filter params (role.id=1182ffc3e9f9477db2003500b7bfd402, scope.project.id=24c8d0395f524641bfe29e37e4b95ba6, user.id=1a817d3534b04f63969a95d3bb500533) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:15:54.479708 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d88a3-bdd7-4ec0-9bcd-fba5e0bf36f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:54.481131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d88a3-bdd7-4ec0-9bcd-fba5e0bf36f2 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:54.486198 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 551/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:15:54 2022] GET /identity/v3/role_assignments?user.id=1a817d3534b04f63969a95d3bb500533&scope.project.id=24c8d0395f524641bfe29e37e4b95ba6&role.id=1182ffc3e9f9477db2003500b7bfd402 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:56.368707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabe01a-1e03-45e3-98a3-eba02294c169 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:56.369218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabe01a-1e03-45e3-98a3-eba02294c169 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:56.369218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabe01a-1e03-45e3-98a3-eba02294c169 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:56.370499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 567/1118] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:56.375654 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaffea-69e0-4e8d-829c-ced59aecd6a0 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:56.376074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaffea-69e0-4e8d-829c-ced59aecd6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:56.376217 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaffea-69e0-4e8d-829c-ced59aecd6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:56.394665 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-9beaffea-69e0-4e8d-829c-ced59aecd6a0 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:56.395879 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9beaffea-69e0-4e8d-829c-ced59aecd6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:56.423984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:56.461202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 552/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:56.730970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabe01a-1e03-45e3-98a3-eba02294c169 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:56.752731 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:56.829604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabe01a-1e03-45e3-98a3-eba02294c169 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:56.861882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275e05e-be85-4b49-b779-d606a7a2d070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:56.862482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275e05e-be85-4b49-b779-d606a7a2d070 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:56.862764 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275e05e-be85-4b49-b779-d606a7a2d070 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:56.863069 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275e05e-be85-4b49-b779-d606a7a2d070 None admin] PATH_INFO: `/v3/roles/1182ffc3e9f9477db2003500b7bfd402` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:56.864450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275e05e-be85-4b49-b779-d606a7a2d070 None admin] RBAC: Authorizing `identity:get_role(role_id=1182ffc3e9f9477db2003500b7bfd402)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:56.865812 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275e05e-be85-4b49-b779-d606a7a2d070 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:56.867831 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 568/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:15:56 2022] GET /identity/v3/roles/1182ffc3e9f9477db2003500b7bfd402 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:56.873331 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beaffea-69e0-4e8d-829c-ced59aecd6a0 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:56.968182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beaffea-69e0-4e8d-829c-ced59aecd6a0 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:57.000354 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf0817-8d2a-4e14-918c-fa3442ba8810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:57.000911 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf0817-8d2a-4e14-918c-fa3442ba8810 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:57.001189 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf0817-8d2a-4e14-918c-fa3442ba8810 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:57.001461 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf0817-8d2a-4e14-918c-fa3442ba8810 None admin] PATH_INFO: `/v3/users/1a817d3534b04f63969a95d3bb500533` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:57.002554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf0817-8d2a-4e14-918c-fa3442ba8810 None admin] RBAC: Authorizing `identity:get_user(user_id=1a817d3534b04f63969a95d3bb500533)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:57.004003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf0817-8d2a-4e14-918c-fa3442ba8810 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:57.005539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 553/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:15:56 2022] GET /identity/v3/users/1a817d3534b04f63969a95d3bb500533 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:57.011599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275e05e-be85-4b49-b779-d606a7a2d070 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:57.096752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:57.108203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275e05e-be85-4b49-b779-d606a7a2d070 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:57.141266 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3380f-fc7c-43b3-9898-6db73c073686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:57.141888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3380f-fc7c-43b3-9898-6db73c073686 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:57.142166 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3380f-fc7c-43b3-9898-6db73c073686 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:57.142434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3380f-fc7c-43b3-9898-6db73c073686 None admin] PATH_INFO: `/v3/projects/24c8d0395f524641bfe29e37e4b95ba6` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:57.143550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3380f-fc7c-43b3-9898-6db73c073686 None admin] RBAC: Authorizing `identity:get_project(project_id=24c8d0395f524641bfe29e37e4b95ba6)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:57.144962 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3380f-fc7c-43b3-9898-6db73c073686 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:57.146565 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 569/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:15:57 2022] GET /identity/v3/projects/24c8d0395f524641bfe29e37e4b95ba6 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:57.153133 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf0817-8d2a-4e14-918c-fa3442ba8810 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:57.186070 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:57.249315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf0817-8d2a-4e14-918c-fa3442ba8810 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:57.282412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743c56d-8a1d-4311-b59c-81040ae8a9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:57.283022 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743c56d-8a1d-4311-b59c-81040ae8a9d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:57.283303 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743c56d-8a1d-4311-b59c-81040ae8a9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:57.283578 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743c56d-8a1d-4311-b59c-81040ae8a9d9 None admin] PATH_INFO: `/v3/projects/24c8d0395f524641bfe29e37e4b95ba6/users/1a817d3534b04f63969a95d3bb500533/roles/1182ffc3e9f9477db2003500b7bfd402` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:57.285661 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743c56d-8a1d-4311-b59c-81040ae8a9d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=24c8d0395f524641bfe29e37e4b95ba6, user_id=1a817d3534b04f63969a95d3bb500533, role_id=1182ffc3e9f9477db2003500b7bfd402)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:57.287111 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743c56d-8a1d-4311-b59c-81040ae8a9d9 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:57.363323 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 554/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:15:57 2022] PUT /identity/v3/projects/24c8d0395f524641bfe29e37e4b95ba6/users/1a817d3534b04f63969a95d3bb500533/roles/1182ffc3e9f9477db2003500b7bfd402 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:15:59.224644 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7d8e6-0db8-4f61-846c-049bee93ea0f None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:59.225182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7d8e6-0db8-4f61-846c-049bee93ea0f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:59.225182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7d8e6-0db8-4f61-846c-049bee93ea0f None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:59.227049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 570/1124] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:15:59.232164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8969d3-4751-48e3-acba-c6eaa7eeb350 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:59.232559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8969d3-4751-48e3-acba-c6eaa7eeb350 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:59.232716 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8969d3-4751-48e3-acba-c6eaa7eeb350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:59.250985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-8a8969d3-4751-48e3-acba-c6eaa7eeb350 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:15:59.252154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8a8969d3-4751-48e3-acba-c6eaa7eeb350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:15:59.320300 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 555/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:15:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:15:59.590542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7d8e6-0db8-4f61-846c-049bee93ea0f None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:59.691398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7d8e6-0db8-4f61-846c-049bee93ea0f None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:59.724339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df503b-2fa7-41ec-9070-5f38cb8cd4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:59.724947 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df503b-2fa7-41ec-9070-5f38cb8cd4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:59.725224 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df503b-2fa7-41ec-9070-5f38cb8cd4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:59.725491 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df503b-2fa7-41ec-9070-5f38cb8cd4c9 None admin] PATH_INFO: `/v3/roles/1182ffc3e9f9477db2003500b7bfd402` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:59.727421 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df503b-2fa7-41ec-9070-5f38cb8cd4c9 None admin] RBAC: Authorizing `identity:get_role(role_id=1182ffc3e9f9477db2003500b7bfd402)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:59.728786 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df503b-2fa7-41ec-9070-5f38cb8cd4c9 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:59.730056 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 571/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:15:59 2022] GET /identity/v3/roles/1182ffc3e9f9477db2003500b7bfd402 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:59.735787 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8969d3-4751-48e3-acba-c6eaa7eeb350 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:59.759197 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:59.829810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8969d3-4751-48e3-acba-c6eaa7eeb350 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:15:59.862633 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6731df9-7c24-4d0c-b7e0-c128bbd171f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:15:59.863225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6731df9-7c24-4d0c-b7e0-c128bbd171f7 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:15:59.863505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6731df9-7c24-4d0c-b7e0-c128bbd171f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:15:59.863778 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6731df9-7c24-4d0c-b7e0-c128bbd171f7 None admin] PATH_INFO: `/v3/users/1a817d3534b04f63969a95d3bb500533` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:15:59.864873 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6731df9-7c24-4d0c-b7e0-c128bbd171f7 None admin] RBAC: Authorizing `identity:get_user(user_id=1a817d3534b04f63969a95d3bb500533)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:15:59.866293 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6731df9-7c24-4d0c-b7e0-c128bbd171f7 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:15:59.868410 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 556/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:15:59 2022] GET /identity/v3/users/1a817d3534b04f63969a95d3bb500533 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:15:59.874200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df503b-2fa7-41ec-9070-5f38cb8cd4c9 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:15:59.899174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:15:59.970365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df503b-2fa7-41ec-9070-5f38cb8cd4c9 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:00.002964 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634920d-a36f-43e5-804f-f9887d91a1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:00.003563 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634920d-a36f-43e5-804f-f9887d91a1ea None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:00.003845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634920d-a36f-43e5-804f-f9887d91a1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:00.004120 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634920d-a36f-43e5-804f-f9887d91a1ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:00.068337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634920d-a36f-43e5-804f-f9887d91a1ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:00.069771 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634920d-a36f-43e5-804f-f9887d91a1ea None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:00.093825 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:00.165844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 572/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:15:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 14:16:00.171771 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6731df9-7c24-4d0c-b7e0-c128bbd171f7 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:00.260135 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:00.268283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6731df9-7c24-4d0c-b7e0-c128bbd171f7 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:00.300552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348dadab-f166-4fa7-b5a6-9de38a7b7a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:00.301168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dadab-f166-4fa7-b5a6-9de38a7b7a2c None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:00.301448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dadab-f166-4fa7-b5a6-9de38a7b7a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:00.301727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dadab-f166-4fa7-b5a6-9de38a7b7a2c None admin] PATH_INFO: `/v3/projects/24c8d0395f524641bfe29e37e4b95ba6` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:00.302856 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348dadab-f166-4fa7-b5a6-9de38a7b7a2c None admin] RBAC: Authorizing `identity:get_project(project_id=24c8d0395f524641bfe29e37e4b95ba6)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:00.304285 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348dadab-f166-4fa7-b5a6-9de38a7b7a2c None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:00.306413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 557/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:16:00 2022] GET /identity/v3/projects/24c8d0395f524641bfe29e37e4b95ba6 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:00.312536 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634920d-a36f-43e5-804f-f9887d91a1ea None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:00.408863 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634920d-a36f-43e5-804f-f9887d91a1ea None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:00.432149 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:00.442604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f196e8-220c-4f9e-b900-7343cddae3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:00.443224 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f196e8-220c-4f9e-b900-7343cddae3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:00.443503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f196e8-220c-4f9e-b900-7343cddae3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:00.443771 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f196e8-220c-4f9e-b900-7343cddae3a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:00.444478 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f196e8-220c-4f9e-b900-7343cddae3a4 None admin] RBAC: Adding query filter params (role.id=1182ffc3e9f9477db2003500b7bfd402, scope.project.id=24c8d0395f524641bfe29e37e4b95ba6, user.id=1a817d3534b04f63969a95d3bb500533) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:16:00.444780 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f196e8-220c-4f9e-b900-7343cddae3a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:00.446815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f196e8-220c-4f9e-b900-7343cddae3a4 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:00.452312 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 573/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:16:00 2022] GET /identity/v3/role_assignments?user.id=1a817d3534b04f63969a95d3bb500533&scope.project.id=24c8d0395f524641bfe29e37e4b95ba6&role.id=1182ffc3e9f9477db2003500b7bfd402 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:02.342915 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae42da-a6e2-4d64-9cf9-307a2f62a74c None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:02.343449 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae42da-a6e2-4d64-9cf9-307a2f62a74c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:02.343449 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae42da-a6e2-4d64-9cf9-307a2f62a74c None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:02.344859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 558/1131] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:16:02.350048 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1f2cd-f433-4197-b4c3-c0675ef6bc3a None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:02.350428 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1f2cd-f433-4197-b4c3-c0675ef6bc3a None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:02.350598 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1f2cd-f433-4197-b4c3-c0675ef6bc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:02.369418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-edf1f2cd-f433-4197-b4c3-c0675ef6bc3a None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:16:02.370597 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-edf1f2cd-f433-4197-b4c3-c0675ef6bc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:16:02.434935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 574/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:16:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:16:02.702902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae42da-a6e2-4d64-9cf9-307a2f62a74c None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:02.800778 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae42da-a6e2-4d64-9cf9-307a2f62a74c None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:02.833923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e19022-1596-41be-bfa6-fdfe90ccd67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:02.834539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e19022-1596-41be-bfa6-fdfe90ccd67d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:02.834844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e19022-1596-41be-bfa6-fdfe90ccd67d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:02.835120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e19022-1596-41be-bfa6-fdfe90ccd67d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:02.836240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e19022-1596-41be-bfa6-fdfe90ccd67d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:02.837672 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e19022-1596-41be-bfa6-fdfe90ccd67d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:02.839242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 559/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:16:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:02.844945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1f2cd-f433-4197-b4c3-c0675ef6bc3a None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:02.941115 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1f2cd-f433-4197-b4c3-c0675ef6bc3a None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:02.958006 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:02.973700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a250284-d9a3-413d-bf80-ed353e70c1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:02.974301 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a250284-d9a3-413d-bf80-ed353e70c1bb None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:02.974584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a250284-d9a3-413d-bf80-ed353e70c1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:02.974905 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a250284-d9a3-413d-bf80-ed353e70c1bb None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:02.975560 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a250284-d9a3-413d-bf80-ed353e70c1bb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:02.977538 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a250284-d9a3-413d-bf80-ed353e70c1bb None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:03.068033 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 575/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 5 14:16:02 2022] POST /identity/v3/projects => generated 311 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:16:04.990697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cfb73-75c8-4630-82a5-58a76dec7ca2 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:04.991281 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cfb73-75c8-4630-82a5-58a76dec7ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:04.991281 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cfb73-75c8-4630-82a5-58a76dec7ca2 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:04.992632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 560/1135] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:16:04.997896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc5090-d4a7-4073-98c6-189198b0ff00 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:04.998283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc5090-d4a7-4073-98c6-189198b0ff00 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:04.998453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc5090-d4a7-4073-98c6-189198b0ff00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:05.016149 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-ebcc5090-d4a7-4073-98c6-189198b0ff00 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:16:05.017340 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ebcc5090-d4a7-4073-98c6-189198b0ff00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:16:05.082098 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 576/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:16:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:16:05.353368 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cfb73-75c8-4630-82a5-58a76dec7ca2 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:05.380357 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:05.451863 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cfb73-75c8-4630-82a5-58a76dec7ca2 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:05.484854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7464f79-7007-470b-8097-5dd02f664659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:05.485476 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7464f79-7007-470b-8097-5dd02f664659 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:05.485757 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7464f79-7007-470b-8097-5dd02f664659 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:05.486075 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7464f79-7007-470b-8097-5dd02f664659 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:05.487215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7464f79-7007-470b-8097-5dd02f664659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:05.488616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7464f79-7007-470b-8097-5dd02f664659 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:05.490160 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 561/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:16:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:05.495903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc5090-d4a7-4073-98c6-189198b0ff00 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:05.589102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:05.591941 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc5090-d4a7-4073-98c6-189198b0ff00 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:05.625018 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6e0f4-594f-4112-90f3-e47662fa9b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:05.625584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6e0f4-594f-4112-90f3-e47662fa9b7d None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:05.625864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6e0f4-594f-4112-90f3-e47662fa9b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:05.626137 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6e0f4-594f-4112-90f3-e47662fa9b7d None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:05.626813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b6e0f4-594f-4112-90f3-e47662fa9b7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:05.628247 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b6e0f4-594f-4112-90f3-e47662fa9b7d None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:05.740462 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 577/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:16:05 2022] POST /identity/v3/users => generated 286 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:16:07.667030 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40457b34-3fad-4a61-8779-8d26efa9281c None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:07.667550 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40457b34-3fad-4a61-8779-8d26efa9281c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:07.667550 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40457b34-3fad-4a61-8779-8d26efa9281c None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:07.668895 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 562/1139] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:16:07.673913 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520f7fd-cb9f-4082-bb3f-bdbd8c3b7aa0 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:07.674296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520f7fd-cb9f-4082-bb3f-bdbd8c3b7aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:07.674471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520f7fd-cb9f-4082-bb3f-bdbd8c3b7aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:07.693043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-f520f7fd-cb9f-4082-bb3f-bdbd8c3b7aa0 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:16:07.694225 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f520f7fd-cb9f-4082-bb3f-bdbd8c3b7aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:16:07.757585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 578/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:16:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:16:08.031999 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40457b34-3fad-4a61-8779-8d26efa9281c None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:08.124092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:08.130484 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40457b34-3fad-4a61-8779-8d26efa9281c None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:08.164308 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ed2cb-9048-4ff9-aa47-39aac2321e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:08.164926 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ed2cb-9048-4ff9-aa47-39aac2321e16 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:08.165209 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ed2cb-9048-4ff9-aa47-39aac2321e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:08.165483 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ed2cb-9048-4ff9-aa47-39aac2321e16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:08.175365 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ed2cb-9048-4ff9-aa47-39aac2321e16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:16:08.180092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-649ed2cb-9048-4ff9-aa47-39aac2321e16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:16:08.180092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:16:08.180092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:16:08.180092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:16:08.180092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:16:08.180092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:16:08.180092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:16:08.180092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:16:08.180092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:16:08.180092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:16:08.180092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:16:08.180092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:16:08.180092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:16:08.180092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:16:08.180092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:16:08.180092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:16:08.181255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:16:08.181255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:16:08.181255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:16:08.181255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:16:08.181255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:16:08.181255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:16:08.181255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:16:08.181255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:16:08.181255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:16:08.181255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:16:08.181255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:16:08.181255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:16:08.181255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:16:08.181255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:16:08.181255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:16:08.181255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:16:08.182343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:16:08.182343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:16:08.182343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:16:08.182343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:16:08.182343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:16:08.182343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:16:08.182343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:16:08.182343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:16:08.182343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:16:08.182343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:16:08.182343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:16:08.182343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:16:08.182343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 563/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:16:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:16:08.187186 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520f7fd-cb9f-4082-bb3f-bdbd8c3b7aa0 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:08.268583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:08.283242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520f7fd-cb9f-4082-bb3f-bdbd8c3b7aa0 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:08.315107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e386b-4317-47d6-836f-6aa06c3c7c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:08.315703 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e386b-4317-47d6-836f-6aa06c3c7c6d None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:08.315984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e386b-4317-47d6-836f-6aa06c3c7c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:08.316255 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e386b-4317-47d6-836f-6aa06c3c7c6d None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:08.316910 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e386b-4317-47d6-836f-6aa06c3c7c6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:16:08.317486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e386b-4317-47d6-836f-6aa06c3c7c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:08.318993 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e386b-4317-47d6-836f-6aa06c3c7c6d None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:08.327590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 579/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:16:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:16:08.333319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ed2cb-9048-4ff9-aa47-39aac2321e16 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:08.428842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ed2cb-9048-4ff9-aa47-39aac2321e16 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:08.461431 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f449bea-d5eb-4765-8546-2e12ba18cc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:08.462037 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f449bea-d5eb-4765-8546-2e12ba18cc51 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:08.462316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f449bea-d5eb-4765-8546-2e12ba18cc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:08.462587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f449bea-d5eb-4765-8546-2e12ba18cc51 None admin] PATH_INFO: `/v3/users/9d259909834844d8b7f46310ad6c4a09` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:08.475109 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f449bea-d5eb-4765-8546-2e12ba18cc51 None admin] RBAC: Authorizing `identity:get_user(user_id=9d259909834844d8b7f46310ad6c4a09)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:08.477081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f449bea-d5eb-4765-8546-2e12ba18cc51 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:08.478636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 564/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:16:08 2022] GET /identity/v3/users/9d259909834844d8b7f46310ad6c4a09 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:08.484443 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e386b-4317-47d6-836f-6aa06c3c7c6d None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:08.580942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e386b-4317-47d6-836f-6aa06c3c7c6d None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:08.612413 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae68241-1426-43e5-b5fb-bbfa10c80108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:08.613003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae68241-1426-43e5-b5fb-bbfa10c80108 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:08.613283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae68241-1426-43e5-b5fb-bbfa10c80108 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:08.613557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae68241-1426-43e5-b5fb-bbfa10c80108 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:08.630651 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:08.678455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae68241-1426-43e5-b5fb-bbfa10c80108 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:08.680466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae68241-1426-43e5-b5fb-bbfa10c80108 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:08.774752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 580/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:16:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:16:08.780516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f449bea-d5eb-4765-8546-2e12ba18cc51 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:08.795573 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:08.877154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f449bea-d5eb-4765-8546-2e12ba18cc51 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:08.909392 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61e5c9-c8fc-42e5-91fd-cf5191582dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:08.910005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e5c9-c8fc-42e5-91fd-cf5191582dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:08.910282 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e5c9-c8fc-42e5-91fd-cf5191582dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:08.910551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e5c9-c8fc-42e5-91fd-cf5191582dd5 None admin] PATH_INFO: `/v3/projects/a06cc4c346f24cd58e83998d4b2c8789` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:08.911667 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61e5c9-c8fc-42e5-91fd-cf5191582dd5 None admin] RBAC: Authorizing `identity:get_project(project_id=a06cc4c346f24cd58e83998d4b2c8789)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:08.913080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61e5c9-c8fc-42e5-91fd-cf5191582dd5 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:08.915248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 565/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:16:08 2022] GET /identity/v3/projects/a06cc4c346f24cd58e83998d4b2c8789 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:08.921194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae68241-1426-43e5-b5fb-bbfa10c80108 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:08.968950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:09.017849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae68241-1426-43e5-b5fb-bbfa10c80108 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:09.050297 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddd1fd-a32f-41a5-9cde-e6f244433094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:09.050910 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddd1fd-a32f-41a5-9cde-e6f244433094 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:09.051188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddd1fd-a32f-41a5-9cde-e6f244433094 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:09.051454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddd1fd-a32f-41a5-9cde-e6f244433094 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:09.052157 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddd1fd-a32f-41a5-9cde-e6f244433094 None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=a06cc4c346f24cd58e83998d4b2c8789, user.id=9d259909834844d8b7f46310ad6c4a09) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:16:09.052457 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddd1fd-a32f-41a5-9cde-e6f244433094 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:09.053841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddd1fd-a32f-41a5-9cde-e6f244433094 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:09.059523 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 581/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:16:08 2022] GET /identity/v3/role_assignments?user.id=9d259909834844d8b7f46310ad6c4a09&scope.project.id=a06cc4c346f24cd58e83998d4b2c8789&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:16:10.923883 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051be2d-75f7-4499-83a4-a46399b79df4 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:10.924415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051be2d-75f7-4499-83a4-a46399b79df4 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:10.924415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051be2d-75f7-4499-83a4-a46399b79df4 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:10.925750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 566/1147] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:16:10.931286 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21978ff1-f3d3-4e43-947a-b51e516750bd None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:10.931717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21978ff1-f3d3-4e43-947a-b51e516750bd None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:10.931816 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21978ff1-f3d3-4e43-947a-b51e516750bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:10.950258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-21978ff1-f3d3-4e43-947a-b51e516750bd None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:16:10.951459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-21978ff1-f3d3-4e43-947a-b51e516750bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:16:11.016652 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 582/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:16:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:16:11.284563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051be2d-75f7-4499-83a4-a46399b79df4 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:11.366056 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:11.384362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051be2d-75f7-4499-83a4-a46399b79df4 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:11.417042 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ca083-5017-4732-bc95-983197c7bf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:11.417658 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ca083-5017-4732-bc95-983197c7bf00 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:11.417938 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ca083-5017-4732-bc95-983197c7bf00 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:11.418212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ca083-5017-4732-bc95-983197c7bf00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:11.427863 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ca083-5017-4732-bc95-983197c7bf00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:16:11.432590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-383ca083-5017-4732-bc95-983197c7bf00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:16:11.432590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:16:11.432590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:16:11.432590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:16:11.432590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:16:11.432590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:16:11.432590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:16:11.432590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:16:11.432590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:16:11.432590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:16:11.432590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:16:11.432590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:16:11.432590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:16:11.432590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:16:11.432590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:16:11.432590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:16:11.433711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:16:11.433711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:16:11.433711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:16:11.433711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:16:11.433711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:16:11.433711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:16:11.433711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:16:11.433711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:16:11.433711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:16:11.433711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:16:11.433711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:16:11.433711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:16:11.433711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:16:11.433711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:16:11.433711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:16:11.433711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:16:11.434745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:16:11.434745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:16:11.434745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:16:11.434745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:16:11.434745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:16:11.434745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:16:11.434745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:16:11.434745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:16:11.434745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:16:11.434745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:16:11.434745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:16:11.434745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:16:11.434745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 567/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:16:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:16:11.439746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21978ff1-f3d3-4e43-947a-b51e516750bd None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:11.481555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:11.535110 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21978ff1-f3d3-4e43-947a-b51e516750bd None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:11.567552 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24af26-111d-481a-94c3-96d1686b643b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:11.568147 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24af26-111d-481a-94c3-96d1686b643b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:11.568427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24af26-111d-481a-94c3-96d1686b643b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:11.568698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24af26-111d-481a-94c3-96d1686b643b None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:11.569351 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24af26-111d-481a-94c3-96d1686b643b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:16:11.569651 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24af26-111d-481a-94c3-96d1686b643b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:11.571627 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24af26-111d-481a-94c3-96d1686b643b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:11.579307 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 583/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:16:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:11.585082 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ca083-5017-4732-bc95-983197c7bf00 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:11.683365 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ca083-5017-4732-bc95-983197c7bf00 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:11.717552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b718d3-5790-41fb-9baa-fde6ba35472d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:11.718165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b718d3-5790-41fb-9baa-fde6ba35472d None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:11.718441 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b718d3-5790-41fb-9baa-fde6ba35472d None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:11.718712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b718d3-5790-41fb-9baa-fde6ba35472d None admin] PATH_INFO: `/v3/users/9d259909834844d8b7f46310ad6c4a09` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:11.719827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b718d3-5790-41fb-9baa-fde6ba35472d None admin] RBAC: Authorizing `identity:get_user(user_id=9d259909834844d8b7f46310ad6c4a09)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:11.721248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b718d3-5790-41fb-9baa-fde6ba35472d None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:11.722802 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 568/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:16:11 2022] GET /identity/v3/users/9d259909834844d8b7f46310ad6c4a09 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:11.728688 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24af26-111d-481a-94c3-96d1686b643b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:11.825645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24af26-111d-481a-94c3-96d1686b643b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:11.827847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:11.857182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb4049-9a4a-4cb9-9b08-c6681f784c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:11.857802 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb4049-9a4a-4cb9-9b08-c6681f784c78 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:11.858082 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb4049-9a4a-4cb9-9b08-c6681f784c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:11.858355 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb4049-9a4a-4cb9-9b08-c6681f784c78 None admin] PATH_INFO: `/v3/projects/a06cc4c346f24cd58e83998d4b2c8789` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:11.859501 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb4049-9a4a-4cb9-9b08-c6681f784c78 None admin] RBAC: Authorizing `identity:get_project(project_id=a06cc4c346f24cd58e83998d4b2c8789)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:11.861249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb4049-9a4a-4cb9-9b08-c6681f784c78 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:11.863425 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 584/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:16:11 2022] GET /identity/v3/projects/a06cc4c346f24cd58e83998d4b2c8789 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:11.869365 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b718d3-5790-41fb-9baa-fde6ba35472d None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:11.881607 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:11.966560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b718d3-5790-41fb-9baa-fde6ba35472d None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:11.999145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba0566-3a19-4104-a3bb-a3762ac82d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:11.999775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba0566-3a19-4104-a3bb-a3762ac82d05 None admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:12.000053 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba0566-3a19-4104-a3bb-a3762ac82d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:12.000322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba0566-3a19-4104-a3bb-a3762ac82d05 None admin] PATH_INFO: `/v3/projects/a06cc4c346f24cd58e83998d4b2c8789/users/9d259909834844d8b7f46310ad6c4a09/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:12.002147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba0566-3a19-4104-a3bb-a3762ac82d05 None admin] RBAC: Authorizing `identity:create_grant(project_id=a06cc4c346f24cd58e83998d4b2c8789, user_id=9d259909834844d8b7f46310ad6c4a09, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:12.003597 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba0566-3a19-4104-a3bb-a3762ac82d05 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:12.072396 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 569/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:16:11 2022] PUT /identity/v3/projects/a06cc4c346f24cd58e83998d4b2c8789/users/9d259909834844d8b7f46310ad6c4a09/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:16:13.925012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995e36e-2f7d-4c24-a25e-565b94c42e82 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:13.925568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995e36e-2f7d-4c24-a25e-565b94c42e82 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:13.925568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995e36e-2f7d-4c24-a25e-565b94c42e82 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:13.926919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 585/1154] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:16:13.932551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109f5bf-4cd9-40d4-9724-a03401f75959 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:13.932992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109f5bf-4cd9-40d4-9724-a03401f75959 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:13.933111 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109f5bf-4cd9-40d4-9724-a03401f75959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:13.951737 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-c109f5bf-4cd9-40d4-9724-a03401f75959 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:16:13.952924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c109f5bf-4cd9-40d4-9724-a03401f75959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:16:14.013710 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:14.018185 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 570/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:16:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:16:14.285066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995e36e-2f7d-4c24-a25e-565b94c42e82 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:14.384724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995e36e-2f7d-4c24-a25e-565b94c42e82 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:14.416662 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed60436-b17c-4c4d-b40f-cca5f7efef0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:14.417278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed60436-b17c-4c4d-b40f-cca5f7efef0a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:14.417555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed60436-b17c-4c4d-b40f-cca5f7efef0a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:14.417824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed60436-b17c-4c4d-b40f-cca5f7efef0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:14.428499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed60436-b17c-4c4d-b40f-cca5f7efef0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:16:14.433737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-fed60436-b17c-4c4d-b40f-cca5f7efef0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:16:14.433737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:16:14.433737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:16:14.433737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:16:14.433737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:16:14.433737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:16:14.433737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:16:14.433737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:16:14.433737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:16:14.433737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:16:14.433737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:16:14.433737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:16:14.433737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:16:14.433737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:16:14.433737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:16:14.433737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:16:14.434895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:16:14.434895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:16:14.434895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:16:14.434895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:16:14.434895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:16:14.434895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:16:14.434895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:16:14.434895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:16:14.434895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:16:14.434895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:16:14.434895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:16:14.434895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:16:14.434895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:16:14.434895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:16:14.434895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:16:14.434895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:16:14.436189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:16:14.436189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:16:14.436189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:16:14.436189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:16:14.436189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:16:14.436189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:16:14.436189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:16:14.436189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:16:14.436189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:16:14.436189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:16:14.436189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:16:14.436189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:16:14.436189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 586/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:16:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:16:14.440778 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109f5bf-4cd9-40d4-9724-a03401f75959 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:14.536545 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109f5bf-4cd9-40d4-9724-a03401f75959 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:14.569597 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62b5d6-8944-4f77-a8cb-3b347b1b305e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:14.570190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62b5d6-8944-4f77-a8cb-3b347b1b305e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:14.570471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62b5d6-8944-4f77-a8cb-3b347b1b305e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:14.570744 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62b5d6-8944-4f77-a8cb-3b347b1b305e None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:14.571461 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62b5d6-8944-4f77-a8cb-3b347b1b305e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:16:14.571763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62b5d6-8944-4f77-a8cb-3b347b1b305e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:14.573151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62b5d6-8944-4f77-a8cb-3b347b1b305e None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:14.580768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 571/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:16:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:14.586773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed60436-b17c-4c4d-b40f-cca5f7efef0a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:14.617614 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:14.683284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed60436-b17c-4c4d-b40f-cca5f7efef0a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:14.716633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d717d-f342-41ec-8577-7aba78ec25a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:14.717262 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d717d-f342-41ec-8577-7aba78ec25a0 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:14.717547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d717d-f342-41ec-8577-7aba78ec25a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:14.717822 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d717d-f342-41ec-8577-7aba78ec25a0 None admin] PATH_INFO: `/v3/users/9d259909834844d8b7f46310ad6c4a09` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:14.718963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d717d-f342-41ec-8577-7aba78ec25a0 None admin] RBAC: Authorizing `identity:get_user(user_id=9d259909834844d8b7f46310ad6c4a09)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:14.720397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d717d-f342-41ec-8577-7aba78ec25a0 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:14.721965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 587/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:16:14 2022] GET /identity/v3/users/9d259909834844d8b7f46310ad6c4a09 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:16:14.728586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62b5d6-8944-4f77-a8cb-3b347b1b305e None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:14.787030 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:14.825242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62b5d6-8944-4f77-a8cb-3b347b1b305e None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:14.857958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37cc80-c0f0-47bf-93d0-0f3d5825ed71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:14.858609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37cc80-c0f0-47bf-93d0-0f3d5825ed71 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:14.859269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37cc80-c0f0-47bf-93d0-0f3d5825ed71 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:14.859550 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37cc80-c0f0-47bf-93d0-0f3d5825ed71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:14.924278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37cc80-c0f0-47bf-93d0-0f3d5825ed71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:14.925720 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37cc80-c0f0-47bf-93d0-0f3d5825ed71 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:14.982883 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:15.020978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 572/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:16:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:16:15.027641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d717d-f342-41ec-8577-7aba78ec25a0 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:15.122942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:15.125888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d717d-f342-41ec-8577-7aba78ec25a0 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:15.159218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbe864-da7e-4b0c-b493-cc8795be7181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:15.159838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbe864-da7e-4b0c-b493-cc8795be7181 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:15.160121 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbe864-da7e-4b0c-b493-cc8795be7181 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:15.160398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbe864-da7e-4b0c-b493-cc8795be7181 None admin] PATH_INFO: `/v3/projects/a06cc4c346f24cd58e83998d4b2c8789` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:15.161513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdbe864-da7e-4b0c-b493-cc8795be7181 None admin] RBAC: Authorizing `identity:get_project(project_id=a06cc4c346f24cd58e83998d4b2c8789)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:15.162972 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdbe864-da7e-4b0c-b493-cc8795be7181 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:15.164607 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 588/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:16:15 2022] GET /identity/v3/projects/a06cc4c346f24cd58e83998d4b2c8789 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:15.170780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37cc80-c0f0-47bf-93d0-0f3d5825ed71 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:15.264969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37cc80-c0f0-47bf-93d0-0f3d5825ed71 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:15.296964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942d878-d217-4213-80a8-804513f0869e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:15.297549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942d878-d217-4213-80a8-804513f0869e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:15.297828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942d878-d217-4213-80a8-804513f0869e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:15.298097 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942d878-d217-4213-80a8-804513f0869e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:15.298827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942d878-d217-4213-80a8-804513f0869e None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=a06cc4c346f24cd58e83998d4b2c8789, user.id=9d259909834844d8b7f46310ad6c4a09) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:16:15.299134 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942d878-d217-4213-80a8-804513f0869e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:15.300564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942d878-d217-4213-80a8-804513f0869e None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:15.306508 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 573/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:16:15 2022] GET /identity/v3/role_assignments?user.id=9d259909834844d8b7f46310ad6c4a09&scope.project.id=a06cc4c346f24cd58e83998d4b2c8789&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:17.198929 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55080744-03ff-4831-a705-616e6143d1b2 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:17.199459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55080744-03ff-4831-a705-616e6143d1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:17.199459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55080744-03ff-4831-a705-616e6143d1b2 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:17.200824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 589/1162] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:16:17.205994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973fbbe-8357-4c68-8acb-a38dbbfeb0fe None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:17.206390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973fbbe-8357-4c68-8acb-a38dbbfeb0fe None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:17.206550 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973fbbe-8357-4c68-8acb-a38dbbfeb0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:17.224775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-2973fbbe-8357-4c68-8acb-a38dbbfeb0fe None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:16:17.225951 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-2973fbbe-8357-4c68-8acb-a38dbbfeb0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:16:17.245188 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:17.289912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 574/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:16:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:16:17.553586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55080744-03ff-4831-a705-616e6143d1b2 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:17.652975 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55080744-03ff-4831-a705-616e6143d1b2 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:17.685220 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf93fad-8c8c-4837-9fe2-9b1eb8aa9236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:17.685817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf93fad-8c8c-4837-9fe2-9b1eb8aa9236 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:17.686102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf93fad-8c8c-4837-9fe2-9b1eb8aa9236 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:17.686379 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf93fad-8c8c-4837-9fe2-9b1eb8aa9236 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:17.719536 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:17.751003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf93fad-8c8c-4837-9fe2-9b1eb8aa9236 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:17.752582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf93fad-8c8c-4837-9fe2-9b1eb8aa9236 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:17.847706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 590/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:16:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:16:17.853537 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973fbbe-8357-4c68-8acb-a38dbbfeb0fe None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:17.948946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973fbbe-8357-4c68-8acb-a38dbbfeb0fe None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:17.981407 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f80c37-713f-437c-a889-406d1f478d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:17.982024 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f80c37-713f-437c-a889-406d1f478d3a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:17.982305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f80c37-713f-437c-a889-406d1f478d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:17.982579 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f80c37-713f-437c-a889-406d1f478d3a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:17.988958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f80c37-713f-437c-a889-406d1f478d3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:17.990393 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f80c37-713f-437c-a889-406d1f478d3a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:18.002609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-36f80c37-713f-437c-a889-406d1f478d3a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 05 14:16:18.002609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:16:18.002609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 05 14:16:18.002609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 05 14:16:18.002609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:16:18.002609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:16:18.002609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:16:18.002609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:16:18.002609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 05 14:16:18.002609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:16:18.002609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:16:18.002609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:16:18.002609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:16:18.002609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:16:18.002609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:16:18.002609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:16:18.002609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:16:18.003817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:16:18.003817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:16:18.003817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:16:18.003817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:16:18.003817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:16:18.003817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:16:18.003817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:16:18.003817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:16:18.003817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:16:18.003817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:16:18.003817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:16:18.003817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:16:18.003817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:16:18.003817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:16:18.003817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:16:18.003817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:16:18.004876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:16:18.004876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:16:18.004876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:16:18.004876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:16:18.004876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:16:18.004876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:16:18.004876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:16:18.004876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:16:18.004876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:16:18.004876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:16:18.004876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 05 14:16:18.004876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:16:18.004876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 05 14:16:18.004876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:16:18.004876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 575/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 5 14:16:17 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:16:18.009889 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf93fad-8c8c-4837-9fe2-9b1eb8aa9236 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:18.074453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:18.106430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf93fad-8c8c-4837-9fe2-9b1eb8aa9236 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:18.139937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b5586-a4f2-4494-ad25-0f11e247d29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:18.140539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b5586-a4f2-4494-ad25-0f11e247d29f None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:18.140824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b5586-a4f2-4494-ad25-0f11e247d29f None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:18.141103 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b5586-a4f2-4494-ad25-0f11e247d29f None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:18.141743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b5586-a4f2-4494-ad25-0f11e247d29f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:16:18.142050 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b5586-a4f2-4494-ad25-0f11e247d29f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:18.143457 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b5586-a4f2-4494-ad25-0f11e247d29f None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:18.150466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 591/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Sep 5 14:16:18 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:20.010417 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e190bca-fbf8-4480-89f2-c16d2f0d2b93 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:20.010997 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e190bca-fbf8-4480-89f2-c16d2f0d2b93 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:20.010997 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e190bca-fbf8-4480-89f2-c16d2f0d2b93 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:20.012332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 576/1167] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:16:20.017624 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302df7b6-77bb-493a-bbc2-41b6e5c240d3 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:20.018007 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302df7b6-77bb-493a-bbc2-41b6e5c240d3 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:20.018176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302df7b6-77bb-493a-bbc2-41b6e5c240d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:20.036712 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-302df7b6-77bb-493a-bbc2-41b6e5c240d3 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:16:20.037891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-302df7b6-77bb-493a-bbc2-41b6e5c240d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:16:20.101697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 592/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:16:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:16:20.368192 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e190bca-fbf8-4480-89f2-c16d2f0d2b93 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:20.386779 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:20.467259 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e190bca-fbf8-4480-89f2-c16d2f0d2b93 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:20.499205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c15159-bb67-490d-93f1-e2a5f0c24ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:20.499800 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c15159-bb67-490d-93f1-e2a5f0c24ead None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:20.500080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c15159-bb67-490d-93f1-e2a5f0c24ead None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:20.500354 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c15159-bb67-490d-93f1-e2a5f0c24ead None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:20.501017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c15159-bb67-490d-93f1-e2a5f0c24ead None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:20.502377 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c15159-bb67-490d-93f1-e2a5f0c24ead None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:20.570618 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 577/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Mon Sep 5 14:16:20 2022] POST /identity/v3/domains => generated 267 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:16:22.483296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41b3e6-2aad-4341-8f88-6cf7613a60fe None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:22.483816 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41b3e6-2aad-4341-8f88-6cf7613a60fe None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:22.483816 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41b3e6-2aad-4341-8f88-6cf7613a60fe None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:22.485174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 593/1170] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:16:22.490312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9704d-687e-4526-b34e-9d026dfd860b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:22.490733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9704d-687e-4526-b34e-9d026dfd860b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:22.491194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9704d-687e-4526-b34e-9d026dfd860b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:22.509237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-bff9704d-687e-4526-b34e-9d026dfd860b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:16:22.510427 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-bff9704d-687e-4526-b34e-9d026dfd860b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:16:22.575767 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 578/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:16:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:16:22.844036 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41b3e6-2aad-4341-8f88-6cf7613a60fe None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:22.904630 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:22.942881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41b3e6-2aad-4341-8f88-6cf7613a60fe None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:22.976754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689104a-d781-44a2-a541-571a17d53f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:22.977385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689104a-d781-44a2-a541-571a17d53f7a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:22.977666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689104a-d781-44a2-a541-571a17d53f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:22.977934 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689104a-d781-44a2-a541-571a17d53f7a None admin] PATH_INFO: `/v3/domains/44137feb8a1748cc829f18e467a6e297` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:22.988139 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689104a-d781-44a2-a541-571a17d53f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=44137feb8a1748cc829f18e467a6e297)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:22.989573 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689104a-d781-44a2-a541-571a17d53f7a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:22.991167 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 594/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Mon Sep 5 14:16:22 2022] GET /identity/v3/domains/44137feb8a1748cc829f18e467a6e297 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:22.997001 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9704d-687e-4526-b34e-9d026dfd860b None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:23.005075 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:23.093819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9704d-687e-4526-b34e-9d026dfd860b None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:23.126921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a36963-4151-499c-ae75-c63f17611c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:23.127523 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a36963-4151-499c-ae75-c63f17611c30 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:23.127807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a36963-4151-499c-ae75-c63f17611c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:23.128084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a36963-4151-499c-ae75-c63f17611c30 None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:23.128726 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a36963-4151-499c-ae75-c63f17611c30 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:23.130160 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a36963-4151-499c-ae75-c63f17611c30 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:23.199424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 579/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 5 14:16:22 2022] POST /identity/v3/projects => generated 361 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:16:25.104603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f11ff6-e537-4ef6-b8c0-d13eb115079e None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:25.105055 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f11ff6-e537-4ef6-b8c0-d13eb115079e None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:25.105164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f11ff6-e537-4ef6-b8c0-d13eb115079e None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:25.106499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 595/1174] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:16:25.111759 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c15dd4-bc51-4b49-ba87-8f19ebe5a07e None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:25.112190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c15dd4-bc51-4b49-ba87-8f19ebe5a07e None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:25.112319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c15dd4-bc51-4b49-ba87-8f19ebe5a07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:25.130131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-78c15dd4-bc51-4b49-ba87-8f19ebe5a07e None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:16:25.131343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-78c15dd4-bc51-4b49-ba87-8f19ebe5a07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:16:25.182469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:25.195623 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 580/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:16:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:16:25.463998 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f11ff6-e537-4ef6-b8c0-d13eb115079e None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:25.563427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f11ff6-e537-4ef6-b8c0-d13eb115079e None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:25.585019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:25.596162 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68afc0-b09e-4d51-89c7-5fef519bfda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:25.596779 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68afc0-b09e-4d51-89c7-5fef519bfda2 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:25.597095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68afc0-b09e-4d51-89c7-5fef519bfda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:25.597359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68afc0-b09e-4d51-89c7-5fef519bfda2 None admin] PATH_INFO: `/v3/domains/44137feb8a1748cc829f18e467a6e297` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:25.598466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68afc0-b09e-4d51-89c7-5fef519bfda2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=44137feb8a1748cc829f18e467a6e297)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:25.599943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68afc0-b09e-4d51-89c7-5fef519bfda2 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:25.601989 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 596/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Mon Sep 5 14:16:25 2022] GET /identity/v3/domains/44137feb8a1748cc829f18e467a6e297 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:25.607658 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c15dd4-bc51-4b49-ba87-8f19ebe5a07e None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:25.702044 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c15dd4-bc51-4b49-ba87-8f19ebe5a07e None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:25.735051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b0283-663d-473f-9da2-41dff6ae73de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:25.735643 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b0283-663d-473f-9da2-41dff6ae73de None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:25.735923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b0283-663d-473f-9da2-41dff6ae73de None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:25.736194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b0283-663d-473f-9da2-41dff6ae73de None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:25.736827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b0283-663d-473f-9da2-41dff6ae73de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:25.738254 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b0283-663d-473f-9da2-41dff6ae73de None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:25.809856 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 581/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:16:25 2022] POST /identity/v3/users => generated 311 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:16:27.742999 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd9ddf-d239-4efc-80b0-16364b477623 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:27.743550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd9ddf-d239-4efc-80b0-16364b477623 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:27.743550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd9ddf-d239-4efc-80b0-16364b477623 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:27.744894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 597/1178] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:16:27.750114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ced557-7398-44dc-87df-2ae3149b1dec None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:27.750544 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ced557-7398-44dc-87df-2ae3149b1dec None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:27.750672 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ced557-7398-44dc-87df-2ae3149b1dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:27.768813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-54ced557-7398-44dc-87df-2ae3149b1dec None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:16:27.769987 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-54ced557-7398-44dc-87df-2ae3149b1dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:16:27.809137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:27.834763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 582/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:16:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:16:28.105255 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd9ddf-d239-4efc-80b0-16364b477623 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:28.202262 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd9ddf-d239-4efc-80b0-16364b477623 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:28.234209 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a37eb8-3d8c-4d22-a77f-41f787278533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:28.234845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a37eb8-3d8c-4d22-a77f-41f787278533 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:28.235129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a37eb8-3d8c-4d22-a77f-41f787278533 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:28.235399 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a37eb8-3d8c-4d22-a77f-41f787278533 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:28.246037 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a37eb8-3d8c-4d22-a77f-41f787278533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:16:28.250733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-c4a37eb8-3d8c-4d22-a77f-41f787278533 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:16:28.250733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:16:28.250733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:16:28.250733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:16:28.250733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:16:28.250733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:16:28.250733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:16:28.250733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:16:28.250733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:16:28.250733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:16:28.250733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:16:28.250733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:16:28.250733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:16:28.250733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:16:28.250733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:16:28.250733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:16:28.251932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:16:28.251932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:16:28.251932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:16:28.251932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:16:28.251932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:16:28.251932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:16:28.251932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:16:28.251932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:16:28.251932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:16:28.251932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:16:28.251932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:16:28.251932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:16:28.251932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:16:28.251932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:16:28.251932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:16:28.251932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:16:28.253072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:16:28.253072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:16:28.253072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:16:28.253072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:16:28.253072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:16:28.253072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:16:28.253072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:16:28.253072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:16:28.253072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:16:28.253072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:16:28.253072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:16:28.253072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:16:28.253072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 598/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:16:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:16:28.258152 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ced557-7398-44dc-87df-2ae3149b1dec None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:28.351922 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ced557-7398-44dc-87df-2ae3149b1dec None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:28.389145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942a820-43d0-4370-b636-c02285024ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:28.389732 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942a820-43d0-4370-b636-c02285024ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:28.390009 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942a820-43d0-4370-b636-c02285024ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:28.390277 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942a820-43d0-4370-b636-c02285024ea1 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:28.390957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942a820-43d0-4370-b636-c02285024ea1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:16:28.391263 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942a820-43d0-4370-b636-c02285024ea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:28.392622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942a820-43d0-4370-b636-c02285024ea1 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:28.400423 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 583/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:16:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:28.406214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a37eb8-3d8c-4d22-a77f-41f787278533 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:28.454745 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:28.502417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a37eb8-3d8c-4d22-a77f-41f787278533 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:28.534274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6a94d-4487-4047-ab0e-f72a35b843b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:28.534917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6a94d-4487-4047-ab0e-f72a35b843b9 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:28.535193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6a94d-4487-4047-ab0e-f72a35b843b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:28.535463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6a94d-4487-4047-ab0e-f72a35b843b9 None admin] PATH_INFO: `/v3/users/8c2fbc29b88e418fb955ece0d5d96655` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:28.548331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6a94d-4487-4047-ab0e-f72a35b843b9 None admin] RBAC: Authorizing `identity:get_user(user_id=8c2fbc29b88e418fb955ece0d5d96655)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:28.549749 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6a94d-4487-4047-ab0e-f72a35b843b9 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:28.551328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 599/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:16:28 2022] GET /identity/v3/users/8c2fbc29b88e418fb955ece0d5d96655 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:28.557739 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942a820-43d0-4370-b636-c02285024ea1 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:28.591971 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:28.653774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942a820-43d0-4370-b636-c02285024ea1 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:28.686539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3e5ea-2d14-4263-8edb-3a2d55a462ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:28.687175 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3e5ea-2d14-4263-8edb-3a2d55a462ea None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:28.687457 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3e5ea-2d14-4263-8edb-3a2d55a462ea None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:28.687729 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3e5ea-2d14-4263-8edb-3a2d55a462ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:28.751756 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d3e5ea-2d14-4263-8edb-3a2d55a462ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:28.753206 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d3e5ea-2d14-4263-8edb-3a2d55a462ea None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:28.784826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:28.846447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 584/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:16:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:16:28.852298 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6a94d-4487-4047-ab0e-f72a35b843b9 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:28.948726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6a94d-4487-4047-ab0e-f72a35b843b9 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:28.957418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:28.981067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4f779-4d48-4f60-a826-347d6a549bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:28.981680 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4f779-4d48-4f60-a826-347d6a549bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:28.981954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4f779-4d48-4f60-a826-347d6a549bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:28.982227 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4f779-4d48-4f60-a826-347d6a549bc1 None admin] PATH_INFO: `/v3/projects/a0801e805c704038a92bac17561b59f8` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:28.983367 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4f779-4d48-4f60-a826-347d6a549bc1 None admin] RBAC: Authorizing `identity:get_project(project_id=a0801e805c704038a92bac17561b59f8)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:28.984925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4f779-4d48-4f60-a826-347d6a549bc1 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:28.987697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 600/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:16:28 2022] GET /identity/v3/projects/a0801e805c704038a92bac17561b59f8 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:16:28.993169 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3e5ea-2d14-4263-8edb-3a2d55a462ea None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:29.088590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3e5ea-2d14-4263-8edb-3a2d55a462ea None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:29.120665 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb90f84-d215-43c2-a1d3-d889f7a84e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:29.121266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb90f84-d215-43c2-a1d3-d889f7a84e33 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:29.121547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb90f84-d215-43c2-a1d3-d889f7a84e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:29.121819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb90f84-d215-43c2-a1d3-d889f7a84e33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:29.122529 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb90f84-d215-43c2-a1d3-d889f7a84e33 None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=a0801e805c704038a92bac17561b59f8, user.id=8c2fbc29b88e418fb955ece0d5d96655) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:16:29.122862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb90f84-d215-43c2-a1d3-d889f7a84e33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:29.124333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb90f84-d215-43c2-a1d3-d889f7a84e33 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:29.127035 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:29.129805 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 585/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:16:28 2022] GET /identity/v3/role_assignments?user.id=8c2fbc29b88e418fb955ece0d5d96655&scope.project.id=a0801e805c704038a92bac17561b59f8&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:30.977649 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f22e1-aeb4-4ac9-882c-f893a8f656ab None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:30.978172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f22e1-aeb4-4ac9-882c-f893a8f656ab None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:30.978172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f22e1-aeb4-4ac9-882c-f893a8f656ab None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:30.979558 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 601/1186] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:16:30.984716 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4962e2-592c-4b00-9493-7c9a6e13e4b8 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:30.985156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4962e2-592c-4b00-9493-7c9a6e13e4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:30.985258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4962e2-592c-4b00-9493-7c9a6e13e4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:31.002873 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-9c4962e2-592c-4b00-9493-7c9a6e13e4b8 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:16:31.004058 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9c4962e2-592c-4b00-9493-7c9a6e13e4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:16:31.068495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 586/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:16:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:16:31.339811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f22e1-aeb4-4ac9-882c-f893a8f656ab None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:31.437328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f22e1-aeb4-4ac9-882c-f893a8f656ab None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:31.470607 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68fa6e-fb80-492a-bd52-7491592c1f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:31.471249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68fa6e-fb80-492a-bd52-7491592c1f6c None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:31.471531 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68fa6e-fb80-492a-bd52-7491592c1f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:31.471801 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68fa6e-fb80-492a-bd52-7491592c1f6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:31.481489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68fa6e-fb80-492a-bd52-7491592c1f6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:16:31.486164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-5f68fa6e-fb80-492a-bd52-7491592c1f6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:16:31.486164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:16:31.486164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:16:31.486164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:16:31.486164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:16:31.486164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:16:31.486164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:16:31.486164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:16:31.486164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:16:31.486164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:16:31.486164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:16:31.486164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:16:31.486164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:16:31.486164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:16:31.486164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:16:31.486164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:16:31.487487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:16:31.487487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:16:31.487487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:16:31.487487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:16:31.487487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:16:31.487487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:16:31.487487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:16:31.487487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:16:31.487487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:16:31.487487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:16:31.487487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:16:31.487487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:16:31.487487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:16:31.487487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:16:31.487487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:16:31.487487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:16:31.488613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:16:31.488613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:16:31.488613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:16:31.488613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:16:31.488613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:16:31.488613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:16:31.488613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:16:31.488613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:16:31.488613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:16:31.488613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:16:31.488613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:16:31.488613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:16:31.488613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 602/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:16:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:16:31.493088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4962e2-592c-4b00-9493-7c9a6e13e4b8 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:31.587750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4962e2-592c-4b00-9493-7c9a6e13e4b8 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:31.614656 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:31.620194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e6122-b179-49c4-905a-7520fe50a3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:31.620791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e6122-b179-49c4-905a-7520fe50a3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:31.621074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e6122-b179-49c4-905a-7520fe50a3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:31.621348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e6122-b179-49c4-905a-7520fe50a3d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:31.622000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e6122-b179-49c4-905a-7520fe50a3d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:16:31.622303 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e6122-b179-49c4-905a-7520fe50a3d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:31.623695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e6122-b179-49c4-905a-7520fe50a3d4 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:31.631343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 587/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:16:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:31.637299 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68fa6e-fb80-492a-bd52-7491592c1f6c None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:31.672188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:31.734542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68fa6e-fb80-492a-bd52-7491592c1f6c None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:31.767108 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd8e10-4926-42dd-8550-6cbf57b9bcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:31.767726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd8e10-4926-42dd-8550-6cbf57b9bcab None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:31.768008 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd8e10-4926-42dd-8550-6cbf57b9bcab None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:31.768282 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd8e10-4926-42dd-8550-6cbf57b9bcab None admin] PATH_INFO: `/v3/users/8c2fbc29b88e418fb955ece0d5d96655` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:31.769389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd8e10-4926-42dd-8550-6cbf57b9bcab None admin] RBAC: Authorizing `identity:get_user(user_id=8c2fbc29b88e418fb955ece0d5d96655)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:31.770832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd8e10-4926-42dd-8550-6cbf57b9bcab None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:31.772386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 603/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:16:31 2022] GET /identity/v3/users/8c2fbc29b88e418fb955ece0d5d96655 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:31.778641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e6122-b179-49c4-905a-7520fe50a3d4 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:31.873453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e6122-b179-49c4-905a-7520fe50a3d4 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:31.904831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0537cbb0-e0f9-4618-a7e8-56a9e1a3f97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:31.905435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537cbb0-e0f9-4618-a7e8-56a9e1a3f97a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:31.905713 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537cbb0-e0f9-4618-a7e8-56a9e1a3f97a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:31.905984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537cbb0-e0f9-4618-a7e8-56a9e1a3f97a None admin] PATH_INFO: `/v3/projects/a0801e805c704038a92bac17561b59f8` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:31.907095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537cbb0-e0f9-4618-a7e8-56a9e1a3f97a None admin] RBAC: Authorizing `identity:get_project(project_id=a0801e805c704038a92bac17561b59f8)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:31.908506 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537cbb0-e0f9-4618-a7e8-56a9e1a3f97a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:31.910065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 588/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:16:31 2022] GET /identity/v3/projects/a0801e805c704038a92bac17561b59f8 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:31.915692 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd8e10-4926-42dd-8550-6cbf57b9bcab None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:32.012091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd8e10-4926-42dd-8550-6cbf57b9bcab None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:32.014375 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:32.044788 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02067b-8a94-4f2a-bfae-c000b7a629da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:32.045441 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02067b-8a94-4f2a-bfae-c000b7a629da None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:32.045723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02067b-8a94-4f2a-bfae-c000b7a629da None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:32.045996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02067b-8a94-4f2a-bfae-c000b7a629da None admin] PATH_INFO: `/v3/projects/a0801e805c704038a92bac17561b59f8/users/8c2fbc29b88e418fb955ece0d5d96655/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:32.047887 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02067b-8a94-4f2a-bfae-c000b7a629da None admin] RBAC: Authorizing `identity:create_grant(project_id=a0801e805c704038a92bac17561b59f8, user_id=8c2fbc29b88e418fb955ece0d5d96655, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:32.049302 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02067b-8a94-4f2a-bfae-c000b7a629da None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:32.095154 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 604/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:16:31 2022] PUT /identity/v3/projects/a0801e805c704038a92bac17561b59f8/users/8c2fbc29b88e418fb955ece0d5d96655/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:16:33.950590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dc6b6-a12e-4df2-a1fa-afd98d2a08cc None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:33.951174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dc6b6-a12e-4df2-a1fa-afd98d2a08cc None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:33.951174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dc6b6-a12e-4df2-a1fa-afd98d2a08cc None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:33.952495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 589/1193] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:16:33.957752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8835b0a-8b04-4b0d-9c62-bd1204d5e1c6 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:33.958190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8835b0a-8b04-4b0d-9c62-bd1204d5e1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:33.958285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8835b0a-8b04-4b0d-9c62-bd1204d5e1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:33.976733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-b8835b0a-8b04-4b0d-9c62-bd1204d5e1c6 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:16:33.977907 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b8835b0a-8b04-4b0d-9c62-bd1204d5e1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:16:34.041524 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 605/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:16:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:16:34.315331 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98dc6b6-a12e-4df2-a1fa-afd98d2a08cc None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:34.374607 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:34.415587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98dc6b6-a12e-4df2-a1fa-afd98d2a08cc None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:34.447767 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba05c8f-7550-411c-8f5a-11beb2fa5194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:34.448383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba05c8f-7550-411c-8f5a-11beb2fa5194 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:34.448664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba05c8f-7550-411c-8f5a-11beb2fa5194 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:34.448938 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba05c8f-7550-411c-8f5a-11beb2fa5194 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:34.459170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba05c8f-7550-411c-8f5a-11beb2fa5194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:16:34.463912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-8ba05c8f-7550-411c-8f5a-11beb2fa5194 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:16:34.463912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:16:34.463912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:16:34.463912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:16:34.463912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:16:34.463912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:16:34.463912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:16:34.463912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:16:34.463912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:16:34.463912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:16:34.463912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:16:34.463912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:16:34.463912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:16:34.463912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:16:34.463912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:16:34.463912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:16:34.465011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:16:34.465011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:16:34.465011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:16:34.465011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:16:34.465011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:16:34.465011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:16:34.465011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:16:34.465011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:16:34.465011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:16:34.465011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:16:34.465011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:16:34.465011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:16:34.465011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:16:34.465011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:16:34.465011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:16:34.465011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:16:34.466081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:16:34.466081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:16:34.466081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:16:34.466081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:16:34.466081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:16:34.466081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:16:34.466081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:16:34.466081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:16:34.466081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:16:34.466081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:16:34.466081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:16:34.466081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:16:34.466081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 590/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:16:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:16:34.471184 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8835b0a-8b04-4b0d-9c62-bd1204d5e1c6 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:34.559544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:34.565848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8835b0a-8b04-4b0d-9c62-bd1204d5e1c6 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:34.599051 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991f815-aeba-4455-8cad-ad75348e86e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:34.599642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991f815-aeba-4455-8cad-ad75348e86e3 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:34.599921 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991f815-aeba-4455-8cad-ad75348e86e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:34.600192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991f815-aeba-4455-8cad-ad75348e86e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:34.601400 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991f815-aeba-4455-8cad-ad75348e86e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:16:34.601701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991f815-aeba-4455-8cad-ad75348e86e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:34.603077 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991f815-aeba-4455-8cad-ad75348e86e3 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:34.610753 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 606/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:16:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:34.616388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba05c8f-7550-411c-8f5a-11beb2fa5194 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:34.712229 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba05c8f-7550-411c-8f5a-11beb2fa5194 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:34.731051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:34.744411 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2907677-a4dd-4f11-b6c2-6374ec4004f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:34.745024 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2907677-a4dd-4f11-b6c2-6374ec4004f8 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:34.745301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2907677-a4dd-4f11-b6c2-6374ec4004f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:34.745571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2907677-a4dd-4f11-b6c2-6374ec4004f8 None admin] PATH_INFO: `/v3/users/8c2fbc29b88e418fb955ece0d5d96655` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:34.746657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2907677-a4dd-4f11-b6c2-6374ec4004f8 None admin] RBAC: Authorizing `identity:get_user(user_id=8c2fbc29b88e418fb955ece0d5d96655)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:34.748103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2907677-a4dd-4f11-b6c2-6374ec4004f8 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:34.749639 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 591/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 5 14:16:34 2022] GET /identity/v3/users/8c2fbc29b88e418fb955ece0d5d96655 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:34.755437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991f815-aeba-4455-8cad-ad75348e86e3 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:34.853893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991f815-aeba-4455-8cad-ad75348e86e3 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:34.885580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b097dd-440b-49a7-89fc-dcec91c4d621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:34.886170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b097dd-440b-49a7-89fc-dcec91c4d621 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:34.886448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b097dd-440b-49a7-89fc-dcec91c4d621 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:34.886716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b097dd-440b-49a7-89fc-dcec91c4d621 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:34.910535 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:34.951655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b097dd-440b-49a7-89fc-dcec91c4d621 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:34.953088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b097dd-440b-49a7-89fc-dcec91c4d621 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:35.047490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 607/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:16:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:16:35.053391 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2907677-a4dd-4f11-b6c2-6374ec4004f8 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:35.150249 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2907677-a4dd-4f11-b6c2-6374ec4004f8 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:35.182135 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bbba2-ab78-4f14-92ba-3584d47e6be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:35.182752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bbba2-ab78-4f14-92ba-3584d47e6be4 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:35.183127 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bbba2-ab78-4f14-92ba-3584d47e6be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:35.183397 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bbba2-ab78-4f14-92ba-3584d47e6be4 None admin] PATH_INFO: `/v3/projects/a0801e805c704038a92bac17561b59f8` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:35.184497 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23bbba2-ab78-4f14-92ba-3584d47e6be4 None admin] RBAC: Authorizing `identity:get_project(project_id=a0801e805c704038a92bac17561b59f8)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:35.185921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23bbba2-ab78-4f14-92ba-3584d47e6be4 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:35.187550 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 592/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 5 14:16:35 2022] GET /identity/v3/projects/a0801e805c704038a92bac17561b59f8 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:16:35.193850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b097dd-440b-49a7-89fc-dcec91c4d621 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:16:35.247896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:35.289434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b097dd-440b-49a7-89fc-dcec91c4d621 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:16:35.321891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06273d-cd7a-4251-9f2e-dc353c7662e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:16:35.322482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06273d-cd7a-4251-9f2e-dc353c7662e9 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:16:35.322756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06273d-cd7a-4251-9f2e-dc353c7662e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:16:35.323051 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06273d-cd7a-4251-9f2e-dc353c7662e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:16:35.323761 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06273d-cd7a-4251-9f2e-dc353c7662e9 None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=a0801e805c704038a92bac17561b59f8, user.id=8c2fbc29b88e418fb955ece0d5d96655) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:16:35.324063 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06273d-cd7a-4251-9f2e-dc353c7662e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:16:35.325465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06273d-cd7a-4251-9f2e-dc353c7662e9 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:16:35.330943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 608/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:16:35 2022] GET /identity/v3/role_assignments?user.id=8c2fbc29b88e418fb955ece0d5d96655&scope.project.id=a0801e805c704038a92bac17561b59f8&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:16:45.177993 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=72 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:16:45.328490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=40 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:17:52.103797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6e337-aead-42f8-8b7c-f795877d1f55 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:52.104261 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6e337-aead-42f8-8b7c-f795877d1f55 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:52.104366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6e337-aead-42f8-8b7c-f795877d1f55 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:52.105703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 593/1201] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:17:52.110778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea9391-99b9-4428-8d3d-e6ea0a7db557 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:52.112360 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea9391-99b9-4428-8d3d-e6ea0a7db557 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:52.112360 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea9391-99b9-4428-8d3d-e6ea0a7db557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:52.129174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-cdea9391-99b9-4428-8d3d-e6ea0a7db557 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:17:52.130339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-cdea9391-99b9-4428-8d3d-e6ea0a7db557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:17:52.195465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 609/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:17:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:17:52.467772 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6e337-aead-42f8-8b7c-f795877d1f55 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:52.565539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6e337-aead-42f8-8b7c-f795877d1f55 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:17:52.599218 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e56ed-bbb6-4cc7-8288-3926e56e1cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:17:52.599866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e56ed-bbb6-4cc7-8288-3926e56e1cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:52.600147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e56ed-bbb6-4cc7-8288-3926e56e1cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:52.600420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e56ed-bbb6-4cc7-8288-3926e56e1cf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:52.610223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e56ed-bbb6-4cc7-8288-3926e56e1cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:17:52.611718 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e56ed-bbb6-4cc7-8288-3926e56e1cf3 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:17:52.626361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-664e56ed-bbb6-4cc7-8288-3926e56e1cf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:17:52.626361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:17:52.626361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:17:52.626361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:17:52.626361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:17:52.626361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:17:52.626361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:17:52.626361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:17:52.626361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:17:52.626361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:17:52.626361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:17:52.626361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:17:52.626361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:17:52.626361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:17:52.626361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:17:52.626361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:17:52.627601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:17:52.627601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:17:52.627601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:17:52.627601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:17:52.627601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:17:52.627601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:17:52.627601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:17:52.627601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:17:52.627601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:17:52.627601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:17:52.627601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:17:52.627601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:17:52.627601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:17:52.627601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:17:52.627601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:17:52.627601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:17:52.627601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:17:52.628862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:17:52.628862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:17:52.628862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:17:52.628862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 594/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:17:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:17:52.633544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea9391-99b9-4428-8d3d-e6ea0a7db557 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:52.730425 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea9391-99b9-4428-8d3d-e6ea0a7db557 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:17:52.758173 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:17:52.763780 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1e955-34cc-43e7-afc3-dc7f100d6703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:17:52.764384 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1e955-34cc-43e7-afc3-dc7f100d6703 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:52.764663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1e955-34cc-43e7-afc3-dc7f100d6703 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:52.764935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1e955-34cc-43e7-afc3-dc7f100d6703 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:52.765574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1e955-34cc-43e7-afc3-dc7f100d6703 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:17:52.765870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1e955-34cc-43e7-afc3-dc7f100d6703 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:17:52.767270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1e955-34cc-43e7-afc3-dc7f100d6703 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:17:52.778059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 610/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:17:52.783828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e56ed-bbb6-4cc7-8288-3926e56e1cf3 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:52.836774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:17:52.882039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e56ed-bbb6-4cc7-8288-3926e56e1cf3 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:17:52.913861 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95dfae-27da-4051-9b62-cff219614f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:17:52.914459 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95dfae-27da-4051-9b62-cff219614f47 None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:52.914735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95dfae-27da-4051-9b62-cff219614f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:52.915034 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95dfae-27da-4051-9b62-cff219614f47 None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:52.915663 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95dfae-27da-4051-9b62-cff219614f47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:17:52.917605 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95dfae-27da-4051-9b62-cff219614f47 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:17:53.067620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 595/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 5 14:17:52 2022] POST /identity/v3/users => generated 251 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:17:54.968693 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee797b45-ee91-40ec-967e-af94d4d50727 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:54.969281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee797b45-ee91-40ec-967e-af94d4d50727 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:54.969281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee797b45-ee91-40ec-967e-af94d4d50727 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:54.970594 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 611/1206] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:17:54.975775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8835a5e-a58b-417b-9e86-0d533848e9aa None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:54.976183 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8835a5e-a58b-417b-9e86-0d533848e9aa None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:54.976333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8835a5e-a58b-417b-9e86-0d533848e9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:54.995566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-b8835a5e-a58b-417b-9e86-0d533848e9aa None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:17:54.996755 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b8835a5e-a58b-417b-9e86-0d533848e9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:17:55.061579 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 596/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:17:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:17:55.333195 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee797b45-ee91-40ec-967e-af94d4d50727 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:55.435472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee797b45-ee91-40ec-967e-af94d4d50727 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:17:55.469841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7a41a-4246-42a6-a055-a5f4677f2e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:17:55.471791 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7a41a-4246-42a6-a055-a5f4677f2e6e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:55.472388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7a41a-4246-42a6-a055-a5f4677f2e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:55.472663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7a41a-4246-42a6-a055-a5f4677f2e6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:55.483836 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7a41a-4246-42a6-a055-a5f4677f2e6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:17:55.490296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-a6f7a41a-4246-42a6-a055-a5f4677f2e6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:17:55.490296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:17:55.490296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:17:55.490296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:17:55.490296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:17:55.490296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:17:55.490296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:17:55.490296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:17:55.490296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:17:55.490296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:17:55.490296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:17:55.490296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:17:55.490296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:17:55.490296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:17:55.490296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:17:55.490296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:17:55.497924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:17:55.497924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:17:55.497924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:17:55.497924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:17:55.497924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:17:55.497924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:17:55.497924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:17:55.497924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:17:55.497924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:17:55.497924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:17:55.497924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:17:55.497924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:17:55.497924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:17:55.497924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:17:55.497924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:17:55.497924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:17:55.499073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:17:55.499073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:17:55.499073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:17:55.499073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:17:55.499073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:17:55.499073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:17:55.499073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:17:55.499073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:17:55.499073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:17:55.499073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:17:55.499073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:17:55.499073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:17:55.499073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 612/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:17:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:17:55.499878 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8835a5e-a58b-417b-9e86-0d533848e9aa None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:55.537772 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:17:55.595389 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8835a5e-a58b-417b-9e86-0d533848e9aa None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:17:55.628055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7056b-1c28-4d3b-8481-7b8b89478804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:17:55.628691 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7056b-1c28-4d3b-8481-7b8b89478804 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:55.628973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7056b-1c28-4d3b-8481-7b8b89478804 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:55.629246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7056b-1c28-4d3b-8481-7b8b89478804 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:55.629901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7056b-1c28-4d3b-8481-7b8b89478804 None admin] RBAC: Adding query filter params (name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:17:55.630201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7056b-1c28-4d3b-8481-7b8b89478804 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:17:55.631716 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7056b-1c28-4d3b-8481-7b8b89478804 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:17:55.639274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 597/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:17:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:17:55.645005 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7a41a-4246-42a6-a055-a5f4677f2e6e None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:55.695447 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:17:55.745830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7a41a-4246-42a6-a055-a5f4677f2e6e None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:17:55.779162 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032902e-0cfa-4d11-a54d-924692dd7561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:17:55.779784 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032902e-0cfa-4d11-a54d-924692dd7561 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:55.780061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032902e-0cfa-4d11-a54d-924692dd7561 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:55.780333 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032902e-0cfa-4d11-a54d-924692dd7561 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:55.790408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032902e-0cfa-4d11-a54d-924692dd7561 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:17:55.791897 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032902e-0cfa-4d11-a54d-924692dd7561 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:17:55.805781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-c032902e-0cfa-4d11-a54d-924692dd7561 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:17:55.805781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:17:55.805781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:17:55.805781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:17:55.805781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:17:55.805781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:17:55.805781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:17:55.805781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:17:55.805781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:17:55.805781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:17:55.805781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:17:55.805781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:17:55.805781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:17:55.805781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:17:55.805781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:17:55.805781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:17:55.807027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:17:55.807027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:17:55.807027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:17:55.807027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:17:55.807027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:17:55.807027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:17:55.807027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:17:55.807027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:17:55.807027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:17:55.807027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:17:55.807027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:17:55.807027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:17:55.807027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:17:55.807027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:17:55.807027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:17:55.807027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:17:55.807027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:17:55.808320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:17:55.808320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:17:55.808320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:17:55.809209 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 613/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:17:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:17:55.813067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7056b-1c28-4d3b-8481-7b8b89478804 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:55.909392 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:17:55.912251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7056b-1c28-4d3b-8481-7b8b89478804 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:17:55.944322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d79dd-99fa-4c23-893a-ea51d73530e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:17:55.944906 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d79dd-99fa-4c23-893a-ea51d73530e3 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:55.945183 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d79dd-99fa-4c23-893a-ea51d73530e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:55.945455 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d79dd-99fa-4c23-893a-ea51d73530e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:55.946079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d79dd-99fa-4c23-893a-ea51d73530e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:17:55.946375 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d79dd-99fa-4c23-893a-ea51d73530e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:17:55.948437 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d79dd-99fa-4c23-893a-ea51d73530e3 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:17:55.959131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 598/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:17:55.965393 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032902e-0cfa-4d11-a54d-924692dd7561 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:56.063241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032902e-0cfa-4d11-a54d-924692dd7561 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:17:56.064799 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:17:56.098616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7219193-2f0e-4e12-9d6c-5494bb7addb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:17:56.099338 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7219193-2f0e-4e12-9d6c-5494bb7addb2 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:56.099622 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7219193-2f0e-4e12-9d6c-5494bb7addb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:56.099888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7219193-2f0e-4e12-9d6c-5494bb7addb2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:56.107234 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7219193-2f0e-4e12-9d6c-5494bb7addb2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:17:56.108702 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7219193-2f0e-4e12-9d6c-5494bb7addb2 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:17:56.120095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-b7219193-2f0e-4e12-9d6c-5494bb7addb2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 14:17:56.120095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:17:56.120095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:17:56.120095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:17:56.120095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:17:56.120095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:17:56.120095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:17:56.120095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:17:56.120095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:17:56.120095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:17:56.120095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:17:56.120095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:17:56.120095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:17:56.120095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:17:56.120095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:17:56.120095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:17:56.125344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:17:56.133397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:17:56.133397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:17:56.133397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:17:56.133397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:17:56.133397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:17:56.133397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:17:56.133397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:17:56.133397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:17:56.133397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 14:17:56.133397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:17:56.133397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 614/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:17:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:17:56.134199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d79dd-99fa-4c23-893a-ea51d73530e3 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:56.225520 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d79dd-99fa-4c23-893a-ea51d73530e3 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:17:56.258213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8696ebc-aa87-4cf7-8803-3e1b13eae9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:17:56.258835 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8696ebc-aa87-4cf7-8803-3e1b13eae9dc None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:56.259118 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8696ebc-aa87-4cf7-8803-3e1b13eae9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:56.259391 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8696ebc-aa87-4cf7-8803-3e1b13eae9dc None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:56.260696 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8696ebc-aa87-4cf7-8803-3e1b13eae9dc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:17:56.261010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8696ebc-aa87-4cf7-8803-3e1b13eae9dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:17:56.262431 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8696ebc-aa87-4cf7-8803-3e1b13eae9dc None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:17:56.278989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 599/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Sep 5 14:17:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:17:56.284835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7219193-2f0e-4e12-9d6c-5494bb7addb2 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:56.382013 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7219193-2f0e-4e12-9d6c-5494bb7addb2 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:17:56.416345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f344d05-b5a4-491c-9ad1-973459318ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:17:56.416943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f344d05-b5a4-491c-9ad1-973459318ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:56.417222 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f344d05-b5a4-491c-9ad1-973459318ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:56.417492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f344d05-b5a4-491c-9ad1-973459318ab3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:56.446966 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:17:56.483103 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f344d05-b5a4-491c-9ad1-973459318ab3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:17:56.484524 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f344d05-b5a4-491c-9ad1-973459318ab3 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:17:56.578878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 615/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:17:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:17:56.584787 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8696ebc-aa87-4cf7-8803-3e1b13eae9dc None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:56.604517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:17:56.683931 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8696ebc-aa87-4cf7-8803-3e1b13eae9dc None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:17:56.716515 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db099d2f-dc59-474e-b7f6-fe8d8618620e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:17:56.717145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db099d2f-dc59-474e-b7f6-fe8d8618620e None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:56.717428 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db099d2f-dc59-474e-b7f6-fe8d8618620e None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:56.717702 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db099d2f-dc59-474e-b7f6-fe8d8618620e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:56.727418 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db099d2f-dc59-474e-b7f6-fe8d8618620e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:17:56.729409 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db099d2f-dc59-474e-b7f6-fe8d8618620e None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:17:56.743659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-db099d2f-dc59-474e-b7f6-fe8d8618620e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:17:56.743659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:17:56.743659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:17:56.743659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:17:56.743659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:17:56.743659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:17:56.743659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:17:56.743659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:17:56.743659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:17:56.743659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:17:56.743659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:17:56.743659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:17:56.743659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:17:56.743659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:17:56.743659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:17:56.743659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:17:56.745315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:17:56.745315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:17:56.745315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:17:56.745315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:17:56.745315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:17:56.745315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:17:56.745315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:17:56.745315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:17:56.745315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:17:56.745315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:17:56.745315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:17:56.745315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:17:56.745315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:17:56.745315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:17:56.745315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:17:56.745315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:17:56.745315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:17:56.746519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:17:56.746519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:17:56.746519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:17:56.746519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 600/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:17:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:17:56.751078 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f344d05-b5a4-491c-9ad1-973459318ab3 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:56.810258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:17:56.846960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f344d05-b5a4-491c-9ad1-973459318ab3 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:17:56.879114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a051ae2-4061-4f21-b258-f3026affbcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:17:56.879812 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a051ae2-4061-4f21-b258-f3026affbcb4 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:56.880035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a051ae2-4061-4f21-b258-f3026affbcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:56.880307 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a051ae2-4061-4f21-b258-f3026affbcb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:56.880977 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a051ae2-4061-4f21-b258-f3026affbcb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:17:56.881277 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a051ae2-4061-4f21-b258-f3026affbcb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:17:56.882634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a051ae2-4061-4f21-b258-f3026affbcb4 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:17:56.894230 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 616/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:17:56.900093 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db099d2f-dc59-474e-b7f6-fe8d8618620e None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:56.968280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:17:56.997780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db099d2f-dc59-474e-b7f6-fe8d8618620e None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:17:57.029392 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f431c-09fa-4d51-9a9a-29609b5c0177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:17:57.029996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f431c-09fa-4d51-9a9a-29609b5c0177 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:57.030274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f431c-09fa-4d51-9a9a-29609b5c0177 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:57.030543 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f431c-09fa-4d51-9a9a-29609b5c0177 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:57.036872 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f431c-09fa-4d51-9a9a-29609b5c0177 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:17:57.038360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f431c-09fa-4d51-9a9a-29609b5c0177 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:17:57.049688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-786f431c-09fa-4d51-9a9a-29609b5c0177 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:17:57.049688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:17:57.049688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:17:57.049688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:17:57.049688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:17:57.049688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:17:57.049688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:17:57.049688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:17:57.049688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:17:57.049688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:17:57.049688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:17:57.049688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:17:57.049688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:17:57.049688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:17:57.049688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:17:57.050876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:17:57.050876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:17:57.050876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:17:57.050876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:17:57.050876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:17:57.050876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:17:57.050876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:17:57.050876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:17:57.050876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:17:57.050876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:17:57.050876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:17:57.050876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:17:57.050876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:17:57.050876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:17:57.050876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:17:57.050876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:17:57.050876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:17:57.052177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:17:57.052177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:17:57.052177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:17:57.052177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:17:57.052177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:17:57.052177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:17:57.052177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:17:57.052177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:17:57.052177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 601/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:17:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:17:57.056589 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a051ae2-4061-4f21-b258-f3026affbcb4 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:57.164400 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a051ae2-4061-4f21-b258-f3026affbcb4 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:17:57.186666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:17:57.201329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349f25f-552d-450d-9255-2e8e72180767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:17:57.201968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349f25f-552d-450d-9255-2e8e72180767 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:57.202280 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349f25f-552d-450d-9255-2e8e72180767 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:57.202556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349f25f-552d-450d-9255-2e8e72180767 None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:57.203281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349f25f-552d-450d-9255-2e8e72180767 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:17:57.203588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349f25f-552d-450d-9255-2e8e72180767 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:17:57.205010 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349f25f-552d-450d-9255-2e8e72180767 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:17:57.217540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 617/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:17:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:17:57.223764 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f431c-09fa-4d51-9a9a-29609b5c0177 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:57.321489 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f431c-09fa-4d51-9a9a-29609b5c0177 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:17:57.349636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:17:57.355318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f1a28-7733-46ea-a56d-204f6abb039f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:17:57.355912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f1a28-7733-46ea-a56d-204f6abb039f None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:57.356193 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f1a28-7733-46ea-a56d-204f6abb039f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:57.356465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f1a28-7733-46ea-a56d-204f6abb039f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:57.357253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f1a28-7733-46ea-a56d-204f6abb039f None admin] RBAC: Adding query filter params (role.id=0a83e40e2d64470a8489bf548e34dabd, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=267825dfb0324be0910203cd5abab377) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:17:57.357572 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f1a28-7733-46ea-a56d-204f6abb039f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:17:57.359002 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f1a28-7733-46ea-a56d-204f6abb039f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:17:57.364123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 602/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:17:57 2022] GET /identity/v3/role_assignments?user.id=267825dfb0324be0910203cd5abab377&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=0a83e40e2d64470a8489bf548e34dabd => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:17:59.238175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cd73b-d627-4fce-b2b2-299d4ef53c78 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:59.238723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cd73b-d627-4fce-b2b2-299d4ef53c78 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:59.238723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cd73b-d627-4fce-b2b2-299d4ef53c78 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:59.240607 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 618/1220] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:17:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:17:59.246431 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a31c53-8b44-4c3b-a8f1-d23e370dcbd0 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:59.246896 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a31c53-8b44-4c3b-a8f1-d23e370dcbd0 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:59.247093 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a31c53-8b44-4c3b-a8f1-d23e370dcbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:59.265866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-91a31c53-8b44-4c3b-a8f1-d23e370dcbd0 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:17:59.267252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-91a31c53-8b44-4c3b-a8f1-d23e370dcbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:17:59.334224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 603/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:17:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:17:59.607393 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cd73b-d627-4fce-b2b2-299d4ef53c78 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:59.711495 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cd73b-d627-4fce-b2b2-299d4ef53c78 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:17:59.744548 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10f9a6-1f74-4e03-8945-71015c4a369d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:17:59.745230 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10f9a6-1f74-4e03-8945-71015c4a369d None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:59.745508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10f9a6-1f74-4e03-8945-71015c4a369d None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:59.745776 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10f9a6-1f74-4e03-8945-71015c4a369d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:59.755650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10f9a6-1f74-4e03-8945-71015c4a369d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:17:59.761014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-6b10f9a6-1f74-4e03-8945-71015c4a369d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:17:59.761014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:17:59.761014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:17:59.761014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:17:59.761014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:17:59.761014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:17:59.761014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:17:59.761014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:17:59.761014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:17:59.761014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:17:59.761014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:17:59.761014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:17:59.761014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:17:59.761014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:17:59.761014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:17:59.761014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:17:59.762708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:17:59.762708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:17:59.762708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:17:59.762708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:17:59.762708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:17:59.762708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:17:59.762708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:17:59.762708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:17:59.762708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:17:59.762708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:17:59.762708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:17:59.762708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:17:59.762708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:17:59.762708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:17:59.762708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:17:59.762708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:17:59.764416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:17:59.764416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:17:59.764416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:17:59.764416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:17:59.764416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:17:59.764416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:17:59.764416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:17:59.764416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:17:59.764416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:17:59.764416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:17:59.764416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:17:59.764416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:17:59.764416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 619/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:17:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 14:17:59.768194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a31c53-8b44-4c3b-a8f1-d23e370dcbd0 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:59.863615 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a31c53-8b44-4c3b-a8f1-d23e370dcbd0 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:17:59.886794 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:17:59.896840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61edbd6-40af-4da7-8577-aa5da19f5b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:17:59.897440 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61edbd6-40af-4da7-8577-aa5da19f5b36 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:17:59.897721 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61edbd6-40af-4da7-8577-aa5da19f5b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:17:59.897994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61edbd6-40af-4da7-8577-aa5da19f5b36 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:17:59.898647 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61edbd6-40af-4da7-8577-aa5da19f5b36 None admin] RBAC: Adding query filter params (name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:17:59.898978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61edbd6-40af-4da7-8577-aa5da19f5b36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:17:59.900358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61edbd6-40af-4da7-8577-aa5da19f5b36 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:17:59.908097 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 604/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:17:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:17:59.913934 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10f9a6-1f74-4e03-8945-71015c4a369d None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:17:59.959128 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:00.012726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10f9a6-1f74-4e03-8945-71015c4a369d None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:00.044503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f841ba3-103b-4d5e-9870-65faf3a15256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:00.045122 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f841ba3-103b-4d5e-9870-65faf3a15256 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:00.045400 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f841ba3-103b-4d5e-9870-65faf3a15256 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:00.045672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f841ba3-103b-4d5e-9870-65faf3a15256 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:00.055226 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f841ba3-103b-4d5e-9870-65faf3a15256 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:00.056656 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f841ba3-103b-4d5e-9870-65faf3a15256 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:00.072182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-6f841ba3-103b-4d5e-9870-65faf3a15256 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:00.072182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:00.072182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:00.072182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:00.072182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:00.072182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:00.072182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:00.072182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:00.072182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:00.072182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:00.072182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:00.072182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:00.072182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:18:00.072182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:18:00.072182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:18:00.072182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:18:00.080870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:00.080870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:00.080870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:00.080870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:00.080870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:00.080870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:00.080870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:00.080870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:00.080870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:00.080870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:00.080870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:00.080870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:00.080870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:00.080870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:00.080870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:00.080870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:00.080870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:18:00.082013 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:18:00.082013 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:00.082013 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:00.082013 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 620/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:17:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:00.082303 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61edbd6-40af-4da7-8577-aa5da19f5b36 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:00.176692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61edbd6-40af-4da7-8577-aa5da19f5b36 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:00.210553 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e1b81-65c5-4581-a239-a9231204383c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:00.211177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e1b81-65c5-4581-a239-a9231204383c None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:00.211458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e1b81-65c5-4581-a239-a9231204383c None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:00.211730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e1b81-65c5-4581-a239-a9231204383c None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:00.212362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e1b81-65c5-4581-a239-a9231204383c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:00.212662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e1b81-65c5-4581-a239-a9231204383c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:00.214248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e1b81-65c5-4581-a239-a9231204383c None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:00.226183 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 605/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:00.232423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f841ba3-103b-4d5e-9870-65faf3a15256 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:00.323507 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:00.329976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f841ba3-103b-4d5e-9870-65faf3a15256 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:00.361938 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245e9ee-2f2a-4ee6-baa0-4b062174c405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:00.362557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245e9ee-2f2a-4ee6-baa0-4b062174c405 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:00.362856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245e9ee-2f2a-4ee6-baa0-4b062174c405 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:00.363128 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245e9ee-2f2a-4ee6-baa0-4b062174c405 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:00.371377 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245e9ee-2f2a-4ee6-baa0-4b062174c405 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:00.376111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245e9ee-2f2a-4ee6-baa0-4b062174c405 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:00.383540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-7245e9ee-2f2a-4ee6-baa0-4b062174c405 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 14:18:00.383540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:00.383540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:00.383540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:00.383540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:00.383540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:00.383540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:00.383540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:00.383540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:00.383540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:00.383540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:00.383540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:00.383540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:18:00.383540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:18:00.383540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:18:00.383540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:00.384672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:00.386017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:00.386017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:00.386017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:18:00.386017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:18:00.386017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:18:00.386017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:18:00.386017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:18:00.386017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:18:00.386017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 14:18:00.386017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:00.386017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 621/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:18:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:00.390597 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e1b81-65c5-4581-a239-a9231204383c None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:00.436826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:00.488131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e1b81-65c5-4581-a239-a9231204383c None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:00.520683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22869c1-04b2-4a23-8bde-401084430e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:00.521283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22869c1-04b2-4a23-8bde-401084430e0a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:00.521572 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22869c1-04b2-4a23-8bde-401084430e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:00.521843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22869c1-04b2-4a23-8bde-401084430e0a None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:00.522557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22869c1-04b2-4a23-8bde-401084430e0a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:00.522975 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22869c1-04b2-4a23-8bde-401084430e0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:00.524403 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22869c1-04b2-4a23-8bde-401084430e0a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:00.541917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 606/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Sep 5 14:18:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:00.547803 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245e9ee-2f2a-4ee6-baa0-4b062174c405 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:00.647750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245e9ee-2f2a-4ee6-baa0-4b062174c405 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:00.680913 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e2711-b73c-405e-a086-19e0b4501002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:00.681533 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e2711-b73c-405e-a086-19e0b4501002 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:00.681815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e2711-b73c-405e-a086-19e0b4501002 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:00.682087 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e2711-b73c-405e-a086-19e0b4501002 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:00.697155 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e2711-b73c-405e-a086-19e0b4501002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:00.697155 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e2711-b73c-405e-a086-19e0b4501002 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:00.707866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-059e2711-b73c-405e-a086-19e0b4501002 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:00.707866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:00.707866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:00.707866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:00.707866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:00.707866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:00.707866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:00.707866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:00.707866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:00.707866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:00.707866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:00.707866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:00.707866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:18:00.707866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:18:00.707866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:18:00.707866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:18:00.709004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:00.709004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:00.709004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:00.709004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:00.709004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:00.709004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:00.709004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:00.709004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:00.709004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:00.709004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:00.709004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:00.709004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:00.709004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:00.709004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:00.709004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:00.709004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:00.709004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:18:00.710091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:18:00.710091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:00.710091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:00.710091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 622/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:18:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:00.715367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22869c1-04b2-4a23-8bde-401084430e0a None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:00.812717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22869c1-04b2-4a23-8bde-401084430e0a None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:00.815020 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:00.845989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d11fc0-3be1-449f-b8eb-9a29c12788c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:00.846592 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d11fc0-3be1-449f-b8eb-9a29c12788c6 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:00.846910 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d11fc0-3be1-449f-b8eb-9a29c12788c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:00.847186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d11fc0-3be1-449f-b8eb-9a29c12788c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:00.847980 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d11fc0-3be1-449f-b8eb-9a29c12788c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:00.848281 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d11fc0-3be1-449f-b8eb-9a29c12788c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:00.849644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d11fc0-3be1-449f-b8eb-9a29c12788c6 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:00.861044 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 607/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:00.866872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e2711-b73c-405e-a086-19e0b4501002 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:00.876024 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:00.965515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e2711-b73c-405e-a086-19e0b4501002 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:00.997949 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f304c-0e04-4c35-a8de-030a324b95ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:00.998568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f304c-0e04-4c35-a8de-030a324b95ce None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:00.998882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f304c-0e04-4c35-a8de-030a324b95ce None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:00.999146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f304c-0e04-4c35-a8de-030a324b95ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:01.005519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f304c-0e04-4c35-a8de-030a324b95ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:01.006968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f304c-0e04-4c35-a8de-030a324b95ce None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:01.018214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-162f304c-0e04-4c35-a8de-030a324b95ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:18:01.018214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:01.018214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:01.018214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:01.018214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:01.018214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:01.018214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:01.018214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:01.018214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:01.018214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:01.018214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:01.018214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:01.018214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:18:01.018214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:18:01.018214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:18:01.020052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:18:01.020052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:01.020052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:01.020052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:01.020052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:01.020052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:01.020052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:01.020052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:01.020052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:01.020052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:01.020052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:01.020052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:01.020052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:01.020052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:01.020052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:01.020052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:01.020052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:01.021190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:18:01.021190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:18:01.021190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:18:01.021190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:18:01.021190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:18:01.021190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:18:01.021190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:18:01.021190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:01.021190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 623/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:18:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:01.025156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d11fc0-3be1-449f-b8eb-9a29c12788c6 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:01.125279 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d11fc0-3be1-449f-b8eb-9a29c12788c6 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:01.159239 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff7a4b-9bc1-4c3c-b1e1-c7d18e1bdb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:01.159837 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff7a4b-9bc1-4c3c-b1e1-c7d18e1bdb12 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:01.160115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff7a4b-9bc1-4c3c-b1e1-c7d18e1bdb12 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:01.160385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff7a4b-9bc1-4c3c-b1e1-c7d18e1bdb12 None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:01.161067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff7a4b-9bc1-4c3c-b1e1-c7d18e1bdb12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:01.161368 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff7a4b-9bc1-4c3c-b1e1-c7d18e1bdb12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:01.162782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff7a4b-9bc1-4c3c-b1e1-c7d18e1bdb12 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:01.174582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 608/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:18:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:01.180605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f304c-0e04-4c35-a8de-030a324b95ce None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:01.244546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:01.278714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f304c-0e04-4c35-a8de-030a324b95ce None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:01.318567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091aa75-f0cd-4497-b765-472ff6ca2f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:01.319546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091aa75-f0cd-4497-b765-472ff6ca2f2a None admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:01.319825 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091aa75-f0cd-4497-b765-472ff6ca2f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:01.320097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091aa75-f0cd-4497-b765-472ff6ca2f2a None admin] PATH_INFO: `/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/267825dfb0324be0910203cd5abab377/roles/0a83e40e2d64470a8489bf548e34dabd` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:01.335322 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1091aa75-f0cd-4497-b765-472ff6ca2f2a None admin] RBAC: Authorizing `identity:create_grant(project_id=d9ca42deaed344eea726fe6e32356efc, user_id=267825dfb0324be0910203cd5abab377, role_id=0a83e40e2d64470a8489bf548e34dabd)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:01.336881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1091aa75-f0cd-4497-b765-472ff6ca2f2a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:01.423714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 624/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:18:01 2022] PUT /identity/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/267825dfb0324be0910203cd5abab377/roles/0a83e40e2d64470a8489bf548e34dabd => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:18:03.289017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a820a-f699-48cd-bde3-6ddff21dadfa None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:03.289539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a820a-f699-48cd-bde3-6ddff21dadfa None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:03.289654 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a820a-f699-48cd-bde3-6ddff21dadfa None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:03.291045 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 609/1233] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:18:03.296728 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590e4b8-1733-4c00-a952-3554ef52d40d None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:03.297113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590e4b8-1733-4c00-a952-3554ef52d40d None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:03.297279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590e4b8-1733-4c00-a952-3554ef52d40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:03.315471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-0590e4b8-1733-4c00-a952-3554ef52d40d None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:18:03.316654 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0590e4b8-1733-4c00-a952-3554ef52d40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:18:03.386845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 625/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:18:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:18:03.663987 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a820a-f699-48cd-bde3-6ddff21dadfa None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:03.688960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:03.761677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a820a-f699-48cd-bde3-6ddff21dadfa None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:03.794318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ea079-9f64-401a-aeb7-392955f888c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:03.795002 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ea079-9f64-401a-aeb7-392955f888c4 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:03.795286 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ea079-9f64-401a-aeb7-392955f888c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:03.795567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ea079-9f64-401a-aeb7-392955f888c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:03.805147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ea079-9f64-401a-aeb7-392955f888c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:18:03.820641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-8e5ea079-9f64-401a-aeb7-392955f888c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:18:03.820641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:03.820641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:03.820641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:03.820641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:03.820641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:03.820641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:03.820641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:03.820641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:03.820641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:03.820641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:03.820641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:03.820641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:18:03.820641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:18:03.820641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:18:03.820641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:18:03.821730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:18:03.821730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:18:03.821730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:18:03.821730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:18:03.821730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:03.821730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:03.821730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:03.821730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:03.821730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:03.821730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:03.821730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:03.821730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:03.821730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:03.821730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:03.821730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:03.821730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:03.822780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:03.822780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:03.822780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:03.822780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:03.822780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:18:03.822780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:18:03.822780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:18:03.822780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:18:03.822780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:18:03.822780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:18:03.822780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 14:18:03.822780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:18:03.822780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 610/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 5 14:18:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:03.823581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590e4b8-1733-4c00-a952-3554ef52d40d None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:03.862715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:03.913583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590e4b8-1733-4c00-a952-3554ef52d40d None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:03.946239 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d7235-7360-4ac4-b53e-482afa4777ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:03.946775 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d7235-7360-4ac4-b53e-482afa4777ac None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:03.947079 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d7235-7360-4ac4-b53e-482afa4777ac None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:03.947348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d7235-7360-4ac4-b53e-482afa4777ac None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:03.947997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d7235-7360-4ac4-b53e-482afa4777ac None admin] RBAC: Adding query filter params (name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:03.948804 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d7235-7360-4ac4-b53e-482afa4777ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:03.950154 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d7235-7360-4ac4-b53e-482afa4777ac None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:03.957965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 626/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 5 14:18:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:03.963798 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ea079-9f64-401a-aeb7-392955f888c4 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:04.047094 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:04.063048 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ea079-9f64-401a-aeb7-392955f888c4 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:04.096328 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d8a3-c5eb-422a-8eac-6f32b1374420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:04.096954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d8a3-c5eb-422a-8eac-6f32b1374420 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:04.097237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d8a3-c5eb-422a-8eac-6f32b1374420 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:04.097511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d8a3-c5eb-422a-8eac-6f32b1374420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:04.107882 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35d8a3-c5eb-422a-8eac-6f32b1374420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:04.109333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35d8a3-c5eb-422a-8eac-6f32b1374420 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:04.123977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-3b35d8a3-c5eb-422a-8eac-6f32b1374420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:04.123977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:04.123977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:04.123977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:04.123977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:04.123977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:04.123977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:04.123977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:04.123977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:04.123977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:04.123977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:04.123977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:04.123977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:18:04.123977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:18:04.123977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:18:04.123977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:18:04.126055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:04.126055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:04.126055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:04.126055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:04.126055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:04.126055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:04.126055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:04.126055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:04.126055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:04.126055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:04.126055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:04.126055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:04.126055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:04.126055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:04.126055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:04.126055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:04.126055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:18:04.127236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:18:04.127236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:04.127236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:18:04.127236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 611/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:04.131200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d7235-7360-4ac4-b53e-482afa4777ac None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:04.227432 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d7235-7360-4ac4-b53e-482afa4777ac None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:04.236441 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:04.260368 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d109dd-ae13-4680-bd21-a9ab2f958143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:04.260994 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d109dd-ae13-4680-bd21-a9ab2f958143 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:04.261289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d109dd-ae13-4680-bd21-a9ab2f958143 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:04.261549 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d109dd-ae13-4680-bd21-a9ab2f958143 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:04.262187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d109dd-ae13-4680-bd21-a9ab2f958143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:04.262489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d109dd-ae13-4680-bd21-a9ab2f958143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:04.263884 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d109dd-ae13-4680-bd21-a9ab2f958143 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:04.275024 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 627/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:04.281299 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d8a3-c5eb-422a-8eac-6f32b1374420 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:04.380227 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d8a3-c5eb-422a-8eac-6f32b1374420 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:04.413614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b33bfe-9bd3-4b1a-8b58-17a3a6b75d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:04.414233 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b33bfe-9bd3-4b1a-8b58-17a3a6b75d58 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:04.414517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b33bfe-9bd3-4b1a-8b58-17a3a6b75d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:04.414818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b33bfe-9bd3-4b1a-8b58-17a3a6b75d58 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:04.418043 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:04.422439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b33bfe-9bd3-4b1a-8b58-17a3a6b75d58 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:04.423934 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b33bfe-9bd3-4b1a-8b58-17a3a6b75d58 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:04.435246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-61b33bfe-9bd3-4b1a-8b58-17a3a6b75d58 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 14:18:04.435246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:04.435246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:04.435246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:04.435246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:04.435246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:04.435246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:04.435246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:04.435246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:04.435246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:04.435246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:04.435246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:04.435246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:18:04.435246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:18:04.435246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:18:04.435246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:04.436510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:04.437683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:04.437683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:04.437683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:18:04.437683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:18:04.437683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:18:04.437683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:18:04.437683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:18:04.437683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:18:04.437683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 14:18:04.437683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:18:04.437683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 612/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:18:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:04.442440 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d109dd-ae13-4680-bd21-a9ab2f958143 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:04.539846 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d109dd-ae13-4680-bd21-a9ab2f958143 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:04.577619 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e927158-c8c2-4d73-b141-bc60f2934700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:04.578220 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e927158-c8c2-4d73-b141-bc60f2934700 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:04.578498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e927158-c8c2-4d73-b141-bc60f2934700 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:04.578769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e927158-c8c2-4d73-b141-bc60f2934700 None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:04.579561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e927158-c8c2-4d73-b141-bc60f2934700 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:04.579868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e927158-c8c2-4d73-b141-bc60f2934700 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:04.581271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e927158-c8c2-4d73-b141-bc60f2934700 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:04.598704 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 628/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Sep 5 14:18:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:04.605154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b33bfe-9bd3-4b1a-8b58-17a3a6b75d58 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:04.701652 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b33bfe-9bd3-4b1a-8b58-17a3a6b75d58 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:04.733876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff25473-a907-43b1-a096-b67cf89c89f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:04.739695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff25473-a907-43b1-a096-b67cf89c89f9 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:04.739695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff25473-a907-43b1-a096-b67cf89c89f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:04.739695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff25473-a907-43b1-a096-b67cf89c89f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:04.793495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:04.799752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff25473-a907-43b1-a096-b67cf89c89f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:04.801192 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff25473-a907-43b1-a096-b67cf89c89f9 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:04.893916 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 613/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:18:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:18:04.899762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e927158-c8c2-4d73-b141-bc60f2934700 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:04.926696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:04.997506 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e927158-c8c2-4d73-b141-bc60f2934700 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:05.031901 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca03c9-aab0-4dda-ae53-0188f48dfb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:05.032534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca03c9-aab0-4dda-ae53-0188f48dfb99 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:05.032795 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca03c9-aab0-4dda-ae53-0188f48dfb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:05.033064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca03c9-aab0-4dda-ae53-0188f48dfb99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:05.042774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca03c9-aab0-4dda-ae53-0188f48dfb99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:05.044653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca03c9-aab0-4dda-ae53-0188f48dfb99 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:05.058727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-3cca03c9-aab0-4dda-ae53-0188f48dfb99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:05.058727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:05.058727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:05.058727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:05.058727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:05.058727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:05.058727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:05.058727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:05.058727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:05.058727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:05.058727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:05.058727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:05.058727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:18:05.058727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:18:05.058727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:18:05.058727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:18:05.059876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:05.059876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:05.059876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:05.059876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:05.059876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:05.059876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:05.059876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:05.059876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:05.059876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:05.059876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:05.059876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:05.059876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:05.059876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:05.059876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:05.059876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:05.059876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:05.059876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:18:05.061012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:18:05.061012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:05.061012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:05.061012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 629/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:18:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:05.065516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff25473-a907-43b1-a096-b67cf89c89f9 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:05.157420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:05.162990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff25473-a907-43b1-a096-b67cf89c89f9 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:05.196115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43cccb-6f1c-4d0d-a062-654431b6c1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:05.196716 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43cccb-6f1c-4d0d-a062-654431b6c1ae None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:05.196996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43cccb-6f1c-4d0d-a062-654431b6c1ae None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:05.197267 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43cccb-6f1c-4d0d-a062-654431b6c1ae None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:05.197894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43cccb-6f1c-4d0d-a062-654431b6c1ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:05.198194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43cccb-6f1c-4d0d-a062-654431b6c1ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:05.200202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43cccb-6f1c-4d0d-a062-654431b6c1ae None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:05.214864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 614/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:05.216580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca03c9-aab0-4dda-ae53-0188f48dfb99 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:05.292693 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:05.313426 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca03c9-aab0-4dda-ae53-0188f48dfb99 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:05.345857 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3375e49-4db1-4940-b44f-fc77fcf8c68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:05.346479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3375e49-4db1-4940-b44f-fc77fcf8c68e None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:05.346761 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3375e49-4db1-4940-b44f-fc77fcf8c68e None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:05.347066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3375e49-4db1-4940-b44f-fc77fcf8c68e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:05.353400 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3375e49-4db1-4940-b44f-fc77fcf8c68e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:05.354861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3375e49-4db1-4940-b44f-fc77fcf8c68e None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:05.365724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-c3375e49-4db1-4940-b44f-fc77fcf8c68e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:18:05.365724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:05.365724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:05.365724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:05.365724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:05.365724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:05.365724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:05.365724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:05.365724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:05.365724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:05.365724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:05.365724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:05.365724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:18:05.365724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:18:05.365724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:18:05.366640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:18:05.366640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:05.366640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:05.366640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:05.366640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:05.366640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:05.366640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:05.366640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:05.366640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:05.366640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:05.366640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:05.366640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:05.366640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:05.366640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:05.366640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:05.366640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:05.366640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:05.368267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:18:05.368267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:18:05.368267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:18:05.368267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:18:05.368267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:18:05.368267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:18:05.368267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:18:05.368267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:05.368267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 630/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:18:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:05.373134 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43cccb-6f1c-4d0d-a062-654431b6c1ae None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:05.470367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43cccb-6f1c-4d0d-a062-654431b6c1ae None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:05.504740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323eb74f-01b5-485e-8a43-e36af28ea021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:05.505330 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323eb74f-01b5-485e-8a43-e36af28ea021 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:05.505608 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323eb74f-01b5-485e-8a43-e36af28ea021 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:05.505876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323eb74f-01b5-485e-8a43-e36af28ea021 None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:05.506550 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323eb74f-01b5-485e-8a43-e36af28ea021 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:05.506876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323eb74f-01b5-485e-8a43-e36af28ea021 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:05.508281 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323eb74f-01b5-485e-8a43-e36af28ea021 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:05.520406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 615/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:18:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:05.526470 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3375e49-4db1-4940-b44f-fc77fcf8c68e None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:05.623555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3375e49-4db1-4940-b44f-fc77fcf8c68e None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:05.654830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb1e5a-439f-4e18-a1f0-ed5bed351dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:05.655426 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb1e5a-439f-4e18-a1f0-ed5bed351dfc None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:05.655706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb1e5a-439f-4e18-a1f0-ed5bed351dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:05.655974 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb1e5a-439f-4e18-a1f0-ed5bed351dfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:05.656687 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb1e5a-439f-4e18-a1f0-ed5bed351dfc None admin] RBAC: Adding query filter params (role.id=0a83e40e2d64470a8489bf548e34dabd, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=267825dfb0324be0910203cd5abab377) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:05.656992 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb1e5a-439f-4e18-a1f0-ed5bed351dfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:05.658400 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb1e5a-439f-4e18-a1f0-ed5bed351dfc None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:05.663952 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 631/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:18:05 2022] GET /identity/v3/role_assignments?user.id=267825dfb0324be0910203cd5abab377&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=0a83e40e2d64470a8489bf548e34dabd => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:07.600123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bb258-a136-4c0b-b210-4cc040ae851f None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:07.600663 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bb258-a136-4c0b-b210-4cc040ae851f None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:07.600791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bb258-a136-4c0b-b210-4cc040ae851f None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:07.602091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 616/1247] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:18:07.607283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14668988-ad4d-4022-a45b-1da04ff20309 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:07.607663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14668988-ad4d-4022-a45b-1da04ff20309 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:07.607834 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14668988-ad4d-4022-a45b-1da04ff20309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:07.614017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:07.626404 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-14668988-ad4d-4022-a45b-1da04ff20309 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:18:07.627612 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-14668988-ad4d-4022-a45b-1da04ff20309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:18:07.692608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 632/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:18:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:18:07.961240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bb258-a136-4c0b-b210-4cc040ae851f None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:07.980052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:08.059646 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bb258-a136-4c0b-b210-4cc040ae851f None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:08.094309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c7d23-915a-4693-bc3d-1471760cd8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:08.094979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c7d23-915a-4693-bc3d-1471760cd8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:08.095262 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c7d23-915a-4693-bc3d-1471760cd8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:08.095539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c7d23-915a-4693-bc3d-1471760cd8e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:08.105445 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c7d23-915a-4693-bc3d-1471760cd8e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:18:08.110258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-552c7d23-915a-4693-bc3d-1471760cd8e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:18:08.110258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:08.110258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:08.110258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:08.110258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:08.110258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:08.110258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:08.110258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:08.110258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:08.110258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:08.110258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:08.110258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:08.110258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:18:08.110258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:18:08.110258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:18:08.110258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:18:08.111385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:18:08.111385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:18:08.111385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:18:08.111385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:18:08.111385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:08.111385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:08.111385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:08.111385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:08.111385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:08.111385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:08.111385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:08.111385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:08.111385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:08.111385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:08.111385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:08.111385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:08.112284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:08.112284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:08.112284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:08.112284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:08.112284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:18:08.112284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:18:08.112284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:18:08.112284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:18:08.112284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:18:08.112284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:18:08.112284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:18:08.112284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:18:08.112284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 617/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:18:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:08.117371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14668988-ad4d-4022-a45b-1da04ff20309 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:08.215052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14668988-ad4d-4022-a45b-1da04ff20309 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:08.242550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:08.248085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99a443-fb1e-4937-bb75-be1cd2ad8e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:08.248736 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99a443-fb1e-4937-bb75-be1cd2ad8e24 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:08.249018 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99a443-fb1e-4937-bb75-be1cd2ad8e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:08.249293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99a443-fb1e-4937-bb75-be1cd2ad8e24 None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:08.249949 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99a443-fb1e-4937-bb75-be1cd2ad8e24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:08.250251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99a443-fb1e-4937-bb75-be1cd2ad8e24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:08.251662 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99a443-fb1e-4937-bb75-be1cd2ad8e24 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:08.259466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 633/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:18:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:08.265756 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c7d23-915a-4693-bc3d-1471760cd8e9 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:08.341138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:08.362909 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c7d23-915a-4693-bc3d-1471760cd8e9 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:08.396557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe39823-229d-41a6-a8a4-804f7c63b90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:08.397182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe39823-229d-41a6-a8a4-804f7c63b90f None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:08.397473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe39823-229d-41a6-a8a4-804f7c63b90f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:08.397734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe39823-229d-41a6-a8a4-804f7c63b90f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:08.407432 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe39823-229d-41a6-a8a4-804f7c63b90f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:08.408871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe39823-229d-41a6-a8a4-804f7c63b90f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:08.423408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-7fe39823-229d-41a6-a8a4-804f7c63b90f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:08.423408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:08.423408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:08.423408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:08.423408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:08.423408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:08.423408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:08.423408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:08.423408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:08.423408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:08.423408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:08.423408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:08.423408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:18:08.423408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:18:08.423408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:18:08.423408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:18:08.424337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:08.424337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:08.424337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:08.424337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:08.424337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:08.424337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:08.424337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:08.424337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:08.424337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:08.424337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:08.424337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:08.424337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:08.424337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:08.424337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:08.424337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:08.424337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:08.424337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:18:08.425348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:18:08.425348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:08.425348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:18:08.425348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 618/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:08.430385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99a443-fb1e-4937-bb75-be1cd2ad8e24 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:08.529585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99a443-fb1e-4937-bb75-be1cd2ad8e24 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:08.562509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080f07d-cc42-40fc-a542-62b1f2116f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:08.563579 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080f07d-cc42-40fc-a542-62b1f2116f83 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:08.563579 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080f07d-cc42-40fc-a542-62b1f2116f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:08.563751 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080f07d-cc42-40fc-a542-62b1f2116f83 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:08.564329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080f07d-cc42-40fc-a542-62b1f2116f83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:08.564629 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080f07d-cc42-40fc-a542-62b1f2116f83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:08.566024 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080f07d-cc42-40fc-a542-62b1f2116f83 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:08.578279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 634/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:08.584119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe39823-229d-41a6-a8a4-804f7c63b90f None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:08.681175 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe39823-229d-41a6-a8a4-804f7c63b90f None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:08.706510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:08.713950 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd6e2e-d783-4ad1-8e49-66889f2aa97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:08.715260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd6e2e-d783-4ad1-8e49-66889f2aa97a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:08.715548 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd6e2e-d783-4ad1-8e49-66889f2aa97a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:08.715819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd6e2e-d783-4ad1-8e49-66889f2aa97a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:08.722965 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd6e2e-d783-4ad1-8e49-66889f2aa97a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:08.724418 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd6e2e-d783-4ad1-8e49-66889f2aa97a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:08.735240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-04fd6e2e-d783-4ad1-8e49-66889f2aa97a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 14:18:08.735240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:08.735240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:08.735240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:08.735240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:08.735240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:08.735240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:08.735240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:08.735240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:08.735240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:08.735240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:08.735240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:08.735240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:18:08.735240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:18:08.735240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:18:08.735240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:08.736339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:08.737687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:08.737687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:08.737687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:18:08.737687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:18:08.737687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:18:08.737687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:18:08.737687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:18:08.737687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:18:08.737687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 14:18:08.737687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:18:08.737687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 619/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:18:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:08.742306 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080f07d-cc42-40fc-a542-62b1f2116f83 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:08.797357 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:08.840024 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080f07d-cc42-40fc-a542-62b1f2116f83 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:08.872192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a33e60-1010-4c6b-84bc-f50a46d0d75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:08.872788 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a33e60-1010-4c6b-84bc-f50a46d0d75b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:08.873070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a33e60-1010-4c6b-84bc-f50a46d0d75b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:08.873343 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a33e60-1010-4c6b-84bc-f50a46d0d75b None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:08.874059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a33e60-1010-4c6b-84bc-f50a46d0d75b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:08.874365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a33e60-1010-4c6b-84bc-f50a46d0d75b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:08.875844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a33e60-1010-4c6b-84bc-f50a46d0d75b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:08.892810 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 635/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Sep 5 14:18:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:08.898628 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd6e2e-d783-4ad1-8e49-66889f2aa97a None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:08.994549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd6e2e-d783-4ad1-8e49-66889f2aa97a None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:09.027212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121de432-62a4-45c1-b2a7-09b2011444fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:09.027814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121de432-62a4-45c1-b2a7-09b2011444fd None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:09.028153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121de432-62a4-45c1-b2a7-09b2011444fd None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:09.028367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121de432-62a4-45c1-b2a7-09b2011444fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:09.082478 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:09.094155 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121de432-62a4-45c1-b2a7-09b2011444fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:09.095646 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121de432-62a4-45c1-b2a7-09b2011444fd None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:09.190122 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 620/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:18:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:18:09.195052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a33e60-1010-4c6b-84bc-f50a46d0d75b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:09.295455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a33e60-1010-4c6b-84bc-f50a46d0d75b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:09.309014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:09.329610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f1e20-b1e4-44ab-833c-cacef8944f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:09.330236 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f1e20-b1e4-44ab-833c-cacef8944f9a None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:09.330555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f1e20-b1e4-44ab-833c-cacef8944f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:09.330860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f1e20-b1e4-44ab-833c-cacef8944f9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:09.340610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f1e20-b1e4-44ab-833c-cacef8944f9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:09.341995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f1e20-b1e4-44ab-833c-cacef8944f9a None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:09.356754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-b80f1e20-b1e4-44ab-833c-cacef8944f9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:09.356754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:09.356754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:09.356754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:09.356754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:09.356754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:09.356754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:09.356754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:09.356754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:09.356754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:09.356754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:09.356754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:09.356754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:18:09.356754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:18:09.356754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:18:09.356754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:18:09.357898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:09.357898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:09.357898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:09.357898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:09.357898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:09.357898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:09.357898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:09.357898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:09.357898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:09.357898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:09.357898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:09.357898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:09.357898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:09.357898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:09.357898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:09.357898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:09.357898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:18:09.359061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:18:09.359061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:09.359061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:09.359061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 636/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:18:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:09.363674 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121de432-62a4-45c1-b2a7-09b2011444fd None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:09.448874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:09.460391 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121de432-62a4-45c1-b2a7-09b2011444fd None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:09.492433 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a8eef-1ee1-4488-b6a0-037ab398ae54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:09.493027 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a8eef-1ee1-4488-b6a0-037ab398ae54 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:09.493310 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a8eef-1ee1-4488-b6a0-037ab398ae54 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:09.493582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a8eef-1ee1-4488-b6a0-037ab398ae54 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:09.494210 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a8eef-1ee1-4488-b6a0-037ab398ae54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:09.494512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a8eef-1ee1-4488-b6a0-037ab398ae54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:09.496490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a8eef-1ee1-4488-b6a0-037ab398ae54 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:09.507192 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 621/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:09.512877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f1e20-b1e4-44ab-833c-cacef8944f9a None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:09.616115 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f1e20-b1e4-44ab-833c-cacef8944f9a None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:09.647928 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb211e-dc89-48cd-a9a9-8455e18d75ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:09.648544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb211e-dc89-48cd-a9a9-8455e18d75ec None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:09.648822 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb211e-dc89-48cd-a9a9-8455e18d75ec None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:09.649093 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb211e-dc89-48cd-a9a9-8455e18d75ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:09.655947 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb211e-dc89-48cd-a9a9-8455e18d75ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:09.657366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb211e-dc89-48cd-a9a9-8455e18d75ec None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:09.668847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-3ffb211e-dc89-48cd-a9a9-8455e18d75ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:18:09.668847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:09.668847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:09.668847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:09.668847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:09.668847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:09.668847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:09.668847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:09.668847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:09.668847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:09.668847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:09.668847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:09.668847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:18:09.668847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:18:09.668847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:18:09.669888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:18:09.669888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:09.669888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:09.669888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:09.669888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:09.669888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:09.669888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:09.669888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:09.669888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:09.669888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:09.669888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:09.669888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:09.669888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:09.669888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:09.669888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:09.669888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:09.669888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:09.671192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:18:09.671192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:18:09.671192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:18:09.671192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:18:09.671192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:18:09.671192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:18:09.671192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:18:09.671192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:09.671192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 637/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:18:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:09.675929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a8eef-1ee1-4488-b6a0-037ab398ae54 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:09.770995 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a8eef-1ee1-4488-b6a0-037ab398ae54 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:09.803372 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274500d-76b5-4146-a7e3-50e55e5ab2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:09.803964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274500d-76b5-4146-a7e3-50e55e5ab2ec None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:09.804243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274500d-76b5-4146-a7e3-50e55e5ab2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:09.804515 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274500d-76b5-4146-a7e3-50e55e5ab2ec None admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:09.805189 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274500d-76b5-4146-a7e3-50e55e5ab2ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:09.805491 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274500d-76b5-4146-a7e3-50e55e5ab2ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:09.806935 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274500d-76b5-4146-a7e3-50e55e5ab2ec None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:09.818356 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 622/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:18:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:09.824728 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb211e-dc89-48cd-a9a9-8455e18d75ec None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:09.850028 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:09.923102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb211e-dc89-48cd-a9a9-8455e18d75ec None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:09.956632 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec2cf3-2fe4-4b75-9e2e-dad73fed51ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:09.957231 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec2cf3-2fe4-4b75-9e2e-dad73fed51ed None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:09.957512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec2cf3-2fe4-4b75-9e2e-dad73fed51ed None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:09.957786 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec2cf3-2fe4-4b75-9e2e-dad73fed51ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:09.958568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec2cf3-2fe4-4b75-9e2e-dad73fed51ed None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=267825dfb0324be0910203cd5abab377) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:09.958900 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec2cf3-2fe4-4b75-9e2e-dad73fed51ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:09.960323 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec2cf3-2fe4-4b75-9e2e-dad73fed51ed None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:09.966584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 638/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:18:09 2022] GET /identity/v3/role_assignments?user.id=267825dfb0324be0910203cd5abab377&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:18:11.820699 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f95fc-aaab-4528-a7c1-220aa39e2cc1 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:11.821233 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f95fc-aaab-4528-a7c1-220aa39e2cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:11.821233 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f95fc-aaab-4528-a7c1-220aa39e2cc1 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:11.822557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 623/1261] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:18:11.828347 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a5ce0-6414-43ae-80a0-ff7a05a6f62b None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:11.828755 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a5ce0-6414-43ae-80a0-ff7a05a6f62b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:11.828866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a5ce0-6414-43ae-80a0-ff7a05a6f62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:11.846976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-d10a5ce0-6414-43ae-80a0-ff7a05a6f62b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:18:11.848214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d10a5ce0-6414-43ae-80a0-ff7a05a6f62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:18:11.913319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 639/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:18:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:18:12.184806 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f95fc-aaab-4528-a7c1-220aa39e2cc1 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:12.195976 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:12.285896 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f95fc-aaab-4528-a7c1-220aa39e2cc1 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:12.319616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88921d2-d530-45a1-aa41-9fe2bc109561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:12.320249 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88921d2-d530-45a1-aa41-9fe2bc109561 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:12.320532 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88921d2-d530-45a1-aa41-9fe2bc109561 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:12.320802 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88921d2-d530-45a1-aa41-9fe2bc109561 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:12.330732 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88921d2-d530-45a1-aa41-9fe2bc109561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:18:12.335448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-d88921d2-d530-45a1-aa41-9fe2bc109561 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:18:12.335448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:12.335448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:12.335448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:12.335448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:12.335448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:12.335448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:12.335448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:12.335448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:12.335448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:12.335448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:12.335448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:12.335448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:18:12.335448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:18:12.335448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:18:12.335448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:18:12.338954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:18:12.338954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:18:12.338954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:18:12.338954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:18:12.338954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:12.338954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:12.338954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:12.338954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:12.338954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:12.338954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:12.338954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:12.338954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:12.338954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:12.338954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:12.338954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:12.338954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:12.339848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:12.339848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:12.339848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:12.339848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:12.339848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:18:12.339848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:18:12.339848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:18:12.339848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:18:12.339848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:18:12.339848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:18:12.339848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:18:12.339848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:18:12.339848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 624/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:18:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:12.342631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a5ce0-6414-43ae-80a0-ff7a05a6f62b None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:12.361886 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:12.440571 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a5ce0-6414-43ae-80a0-ff7a05a6f62b None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:12.473315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaed588-f9d0-4111-a916-14ae2424912b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:12.473924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaed588-f9d0-4111-a916-14ae2424912b None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:12.474205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaed588-f9d0-4111-a916-14ae2424912b None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:12.474477 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaed588-f9d0-4111-a916-14ae2424912b None admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:12.475169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaed588-f9d0-4111-a916-14ae2424912b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:12.475472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaed588-f9d0-4111-a916-14ae2424912b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:12.476879 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaed588-f9d0-4111-a916-14ae2424912b None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:12.485454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 640/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:18:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:12.491459 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88921d2-d530-45a1-aa41-9fe2bc109561 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:12.558286 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:12.591617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88921d2-d530-45a1-aa41-9fe2bc109561 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:12.625796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a2568-e301-4801-97f0-e34e77c8f432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:12.626359 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a2568-e301-4801-97f0-e34e77c8f432 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:12.626644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a2568-e301-4801-97f0-e34e77c8f432 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:12.626949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a2568-e301-4801-97f0-e34e77c8f432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:12.637422 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a2568-e301-4801-97f0-e34e77c8f432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:12.638900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a2568-e301-4801-97f0-e34e77c8f432 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:12.653675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-ca8a2568-e301-4801-97f0-e34e77c8f432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:12.653675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:12.653675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:12.653675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:12.653675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:12.653675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:12.653675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:12.653675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:12.653675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:12.653675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:12.653675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:12.653675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:12.653675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:18:12.653675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:18:12.653675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:18:12.653675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:18:12.655673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:12.655673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:12.655673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:12.655673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:12.655673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:12.655673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:12.655673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:12.655673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:12.655673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:12.655673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:12.655673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:12.655673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:12.655673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:12.655673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:12.655673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:12.655673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:12.655673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:18:12.656603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:18:12.656603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:12.656603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:18:12.656603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 625/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:18:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:12.660617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaed588-f9d0-4111-a916-14ae2424912b None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:12.741717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:12.760599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaed588-f9d0-4111-a916-14ae2424912b None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:12.793134 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283060bb-be73-4e33-88ab-7778886c0dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:12.793772 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283060bb-be73-4e33-88ab-7778886c0dca None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:12.794049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283060bb-be73-4e33-88ab-7778886c0dca None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:12.794316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283060bb-be73-4e33-88ab-7778886c0dca None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:12.794975 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283060bb-be73-4e33-88ab-7778886c0dca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:12.795276 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283060bb-be73-4e33-88ab-7778886c0dca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:12.796658 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283060bb-be73-4e33-88ab-7778886c0dca None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:12.809056 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 641/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:12.814746 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a2568-e301-4801-97f0-e34e77c8f432 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:12.911969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a2568-e301-4801-97f0-e34e77c8f432 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:12.931231 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:12.945813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16a725-0d9e-4c2d-b08b-23d302993de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:12.946437 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16a725-0d9e-4c2d-b08b-23d302993de9 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:12.946721 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16a725-0d9e-4c2d-b08b-23d302993de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:12.947029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16a725-0d9e-4c2d-b08b-23d302993de9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:12.954241 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16a725-0d9e-4c2d-b08b-23d302993de9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:12.955721 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16a725-0d9e-4c2d-b08b-23d302993de9 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:12.966600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-4c16a725-0d9e-4c2d-b08b-23d302993de9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 14:18:12.966600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:12.966600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:12.966600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:12.966600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:12.966600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:12.966600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:12.966600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:12.966600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:12.966600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:12.966600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:12.966600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:12.966600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:18:12.966600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:18:12.966600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:18:12.966600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:12.967826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:12.970142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:12.970142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:12.970142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:18:12.970142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:18:12.970142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:18:12.970142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:18:12.970142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:18:12.970142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:18:12.970142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 14:18:12.970142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:18:12.970142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 626/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:18:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:12.973573 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283060bb-be73-4e33-88ab-7778886c0dca None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:13.070289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283060bb-be73-4e33-88ab-7778886c0dca None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:13.105461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249ae62-9421-4845-9ecf-12fe87b88006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:13.106094 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249ae62-9421-4845-9ecf-12fe87b88006 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:13.106375 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249ae62-9421-4845-9ecf-12fe87b88006 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:13.106649 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249ae62-9421-4845-9ecf-12fe87b88006 None admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:13.107420 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249ae62-9421-4845-9ecf-12fe87b88006 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:13.107803 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249ae62-9421-4845-9ecf-12fe87b88006 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:13.109172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249ae62-9421-4845-9ecf-12fe87b88006 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:13.117820 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:13.126693 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 642/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Sep 5 14:18:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:13.132332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16a725-0d9e-4c2d-b08b-23d302993de9 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:13.233309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16a725-0d9e-4c2d-b08b-23d302993de9 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:13.265677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b499531-ab3a-4687-a415-1501797e789a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:13.266292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b499531-ab3a-4687-a415-1501797e789a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:13.266569 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b499531-ab3a-4687-a415-1501797e789a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:13.266867 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b499531-ab3a-4687-a415-1501797e789a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:13.277406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b499531-ab3a-4687-a415-1501797e789a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:13.278869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b499531-ab3a-4687-a415-1501797e789a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:13.292297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-3b499531-ab3a-4687-a415-1501797e789a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:13.292297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:13.292297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:13.292297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:13.292297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:13.292297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:13.292297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:13.292297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:13.292297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:13.292297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:13.292297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:13.292297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:13.292297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:18:13.292297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:18:13.292297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:18:13.292297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:18:13.294028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:13.294028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:13.294028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:13.294028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:13.294028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:13.294028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:13.294028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:13.294028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:13.294028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:13.294028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:13.294028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:13.294028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:13.294028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:13.294028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:13.294028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:13.294028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:13.294028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:18:13.295259 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:18:13.295259 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:13.295259 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:18:13.298698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 627/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:13.299903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249ae62-9421-4845-9ecf-12fe87b88006 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:13.395613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249ae62-9421-4845-9ecf-12fe87b88006 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:13.430177 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c285bc7-e2a2-43c3-80ef-cf0e65361f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:13.430823 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c285bc7-e2a2-43c3-80ef-cf0e65361f46 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:13.431108 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c285bc7-e2a2-43c3-80ef-cf0e65361f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:13.431381 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c285bc7-e2a2-43c3-80ef-cf0e65361f46 None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:13.432020 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c285bc7-e2a2-43c3-80ef-cf0e65361f46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:13.432321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c285bc7-e2a2-43c3-80ef-cf0e65361f46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:13.433759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c285bc7-e2a2-43c3-80ef-cf0e65361f46 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:13.444872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 643/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:13.450549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b499531-ab3a-4687-a415-1501797e789a None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:13.481836 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:13.553293 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b499531-ab3a-4687-a415-1501797e789a None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:13.586906 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8162d-3d3c-4e84-8d72-8933f61702dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:13.587600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8162d-3d3c-4e84-8d72-8933f61702dc None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:13.587890 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8162d-3d3c-4e84-8d72-8933f61702dc None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:13.588167 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8162d-3d3c-4e84-8d72-8933f61702dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:13.595233 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8162d-3d3c-4e84-8d72-8933f61702dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:13.596674 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8162d-3d3c-4e84-8d72-8933f61702dc None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:13.608436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-22a8162d-3d3c-4e84-8d72-8933f61702dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:18:13.608436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:13.608436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:13.608436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:13.608436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:13.608436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:13.608436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:13.608436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:13.608436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:13.608436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:13.608436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:13.608436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:13.608436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:18:13.608436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:18:13.608436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:18:13.609559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:18:13.609559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:13.609559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:13.609559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:13.609559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:13.609559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:13.609559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:13.609559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:13.609559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:13.609559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:13.609559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:13.609559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:13.609559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:13.609559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:13.609559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:13.609559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:13.609559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:13.610662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:18:13.610662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:18:13.610662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:18:13.610662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:18:13.610662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:18:13.610662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:18:13.610662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:18:13.610662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:18:13.610662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 628/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:18:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:13.615731 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c285bc7-e2a2-43c3-80ef-cf0e65361f46 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:13.672474 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:13.712550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c285bc7-e2a2-43c3-80ef-cf0e65361f46 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:13.745357 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bd2ea-2013-428a-a841-6f8cdc941d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:13.745985 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bd2ea-2013-428a-a841-6f8cdc941d74 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:13.746267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bd2ea-2013-428a-a841-6f8cdc941d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:13.746541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bd2ea-2013-428a-a841-6f8cdc941d74 None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:13.747250 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bd2ea-2013-428a-a841-6f8cdc941d74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:13.747557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bd2ea-2013-428a-a841-6f8cdc941d74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:13.748967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bd2ea-2013-428a-a841-6f8cdc941d74 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:13.760519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 644/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:18:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:13.767103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8162d-3d3c-4e84-8d72-8933f61702dc None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:13.856308 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:13.864451 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8162d-3d3c-4e84-8d72-8933f61702dc None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:13.896720 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efc626-a5cf-4987-ac79-2839191df21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:13.897366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efc626-a5cf-4987-ac79-2839191df21f None admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:13.897648 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efc626-a5cf-4987-ac79-2839191df21f None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:13.897919 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efc626-a5cf-4987-ac79-2839191df21f None admin] PATH_INFO: `/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/267825dfb0324be0910203cd5abab377/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:13.899813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7efc626-a5cf-4987-ac79-2839191df21f None admin] RBAC: Authorizing `identity:create_grant(project_id=d9ca42deaed344eea726fe6e32356efc, user_id=267825dfb0324be0910203cd5abab377, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:13.901228 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7efc626-a5cf-4987-ac79-2839191df21f None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:13.997600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 629/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 5 14:18:13 2022] PUT /identity/v3/projects/d9ca42deaed344eea726fe6e32356efc/users/267825dfb0324be0910203cd5abab377/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:18:15.886280 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970969e7-cd28-4a31-a814-9bd8599c38c0 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:15.886933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970969e7-cd28-4a31-a814-9bd8599c38c0 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:15.886933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970969e7-cd28-4a31-a814-9bd8599c38c0 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:15.888185 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 645/1274] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:18:15.893501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216db734-398b-43c0-bc10-c748e94a29e0 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:15.893920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216db734-398b-43c0-bc10-c748e94a29e0 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:15.894048 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216db734-398b-43c0-bc10-c748e94a29e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:15.912330 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-216db734-398b-43c0-bc10-c748e94a29e0 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:18:15.913597 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-216db734-398b-43c0-bc10-c748e94a29e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:18:15.979294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 630/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:18:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:18:16.249611 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970969e7-cd28-4a31-a814-9bd8599c38c0 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:16.354734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970969e7-cd28-4a31-a814-9bd8599c38c0 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:16.372257 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:16.388988 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c74d79-a9bf-424f-81ce-4e28f40ef3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:16.389601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c74d79-a9bf-424f-81ce-4e28f40ef3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:16.389878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c74d79-a9bf-424f-81ce-4e28f40ef3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:16.390149 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c74d79-a9bf-424f-81ce-4e28f40ef3b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:16.399895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c74d79-a9bf-424f-81ce-4e28f40ef3b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 14:18:16.404623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-c9c74d79-a9bf-424f-81ce-4e28f40ef3b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:18:16.404623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:16.404623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:16.404623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:16.404623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:16.404623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:16.404623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:16.404623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:16.404623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:16.404623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:16.404623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:16.404623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:16.404623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 14:18:16.404623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 14:18:16.404623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 14:18:16.404623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 14:18:16.405674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 14:18:16.405674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 14:18:16.405674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 14:18:16.405674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 14:18:16.405674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:16.405674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:16.405674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:16.405674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:16.405674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:16.405674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:16.405674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:16.405674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:16.405674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:16.405674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:16.405674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:16.405674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:16.406546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:16.406546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:16.406546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:16.406546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:16.406546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 14:18:16.406546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 14:18:16.406546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 14:18:16.406546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 14:18:16.406546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 14:18:16.406546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 14:18:16.406546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 14:18:16.406546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:16.406546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 646/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 5 14:18:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:16.411941 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216db734-398b-43c0-bc10-c748e94a29e0 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:16.490375 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:16.508023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216db734-398b-43c0-bc10-c748e94a29e0 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:16.540659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9225c4-2766-4e92-8606-25f7ae074551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:16.541254 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9225c4-2766-4e92-8606-25f7ae074551 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:16.541538 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9225c4-2766-4e92-8606-25f7ae074551 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:16.541879 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9225c4-2766-4e92-8606-25f7ae074551 None admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:16.542540 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9225c4-2766-4e92-8606-25f7ae074551 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:16.542864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9225c4-2766-4e92-8606-25f7ae074551 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:16.544231 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9225c4-2766-4e92-8606-25f7ae074551 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:16.551909 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 631/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 5 14:18:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:16.557665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c74d79-a9bf-424f-81ce-4e28f40ef3b2 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:16.656581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c74d79-a9bf-424f-81ce-4e28f40ef3b2 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:16.690008 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbe314-742d-4d02-bf5e-92eafadda319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:16.690655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbe314-742d-4d02-bf5e-92eafadda319 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:16.690967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbe314-742d-4d02-bf5e-92eafadda319 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:16.691271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbe314-742d-4d02-bf5e-92eafadda319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:16.701418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbe314-742d-4d02-bf5e-92eafadda319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:16.702875 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbe314-742d-4d02-bf5e-92eafadda319 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:16.717096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-17fbe314-742d-4d02-bf5e-92eafadda319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:16.717096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:16.717096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:16.717096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:16.717096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:16.717096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:16.717096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:16.717096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:16.717096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:16.717096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:16.717096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:16.717096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:16.717096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:18:16.717096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:18:16.717096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:18:16.717096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:18:16.721165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:16.721165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:16.721165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:16.721165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:16.721165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:16.721165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:16.721165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:16.721165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:16.721165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:16.721165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:16.721165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:16.721165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:16.721165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:16.721165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:16.721165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:16.721165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:16.721165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:18:16.722316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:18:16.722316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:16.722316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:16.722316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 647/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:18:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:16.724342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9225c4-2766-4e92-8606-25f7ae074551 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:16.820977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9225c4-2766-4e92-8606-25f7ae074551 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:16.853494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd7bb9-526e-414d-836b-66f30dd80579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:16.854094 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd7bb9-526e-414d-836b-66f30dd80579 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:16.854374 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd7bb9-526e-414d-836b-66f30dd80579 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:16.854647 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd7bb9-526e-414d-836b-66f30dd80579 None admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:16.855828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd7bb9-526e-414d-836b-66f30dd80579 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:16.856130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd7bb9-526e-414d-836b-66f30dd80579 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:16.857496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd7bb9-526e-414d-836b-66f30dd80579 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:16.868539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 632/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:16.874848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbe314-742d-4d02-bf5e-92eafadda319 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:16.904962 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:16.972332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbe314-742d-4d02-bf5e-92eafadda319 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:17.004970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a48053-7b75-4c05-8892-41fdcd5aad6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:17.005586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a48053-7b75-4c05-8892-41fdcd5aad6f None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:17.005884 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a48053-7b75-4c05-8892-41fdcd5aad6f None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:17.007139 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a48053-7b75-4c05-8892-41fdcd5aad6f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:17.015926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a48053-7b75-4c05-8892-41fdcd5aad6f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:17.017362 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a48053-7b75-4c05-8892-41fdcd5aad6f None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:17.028995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-97a48053-7b75-4c05-8892-41fdcd5aad6f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 14:18:17.028995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:17.028995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:17.028995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:17.028995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:17.028995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:17.028995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:17.028995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:17.028995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:17.028995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:17.028995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:17.028995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:17.028995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 14:18:17.028995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 14:18:17.028995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 14:18:17.028995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:17.034833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:17.035890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:17.035890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:17.035890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 14:18:17.035890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 14:18:17.035890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 14:18:17.035890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 14:18:17.035890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 14:18:17.035890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 14:18:17.035890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 14:18:17.035890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:17.035890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 648/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:18:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:17.036518 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd7bb9-526e-414d-836b-66f30dd80579 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:17.039876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:17.138402 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd7bb9-526e-414d-836b-66f30dd80579 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:17.170650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c19f7-3986-404b-b93d-88d3a556439b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:17.171330 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c19f7-3986-404b-b93d-88d3a556439b None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:17.171611 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c19f7-3986-404b-b93d-88d3a556439b None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:17.171885 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c19f7-3986-404b-b93d-88d3a556439b None admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:17.172605 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c19f7-3986-404b-b93d-88d3a556439b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:17.172910 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c19f7-3986-404b-b93d-88d3a556439b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:17.174371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c19f7-3986-404b-b93d-88d3a556439b None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:17.192158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 633/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Sep 5 14:18:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:17.198345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a48053-7b75-4c05-8892-41fdcd5aad6f None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:17.287340 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:17.297190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a48053-7b75-4c05-8892-41fdcd5aad6f None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:17.331111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04208275-5c0f-4ae1-a1f6-aadc451af1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:17.331713 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04208275-5c0f-4ae1-a1f6-aadc451af1eb None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:17.331991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04208275-5c0f-4ae1-a1f6-aadc451af1eb None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:17.332293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04208275-5c0f-4ae1-a1f6-aadc451af1eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:17.397519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04208275-5c0f-4ae1-a1f6-aadc451af1eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:17.398917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04208275-5c0f-4ae1-a1f6-aadc451af1eb None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:17.483564 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:17.493381 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 649/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 5 14:18:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 14:18:17.499192 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c19f7-3986-404b-b93d-88d3a556439b None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:17.558570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:17.600119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c19f7-3986-404b-b93d-88d3a556439b None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:17.631585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8b2cb-ce84-4010-b9d2-5db37f3745e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:17.632203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8b2cb-ce84-4010-b9d2-5db37f3745e2 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:17.632482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8b2cb-ce84-4010-b9d2-5db37f3745e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:17.632750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8b2cb-ce84-4010-b9d2-5db37f3745e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:17.643237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8b2cb-ce84-4010-b9d2-5db37f3745e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:17.644637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8b2cb-ce84-4010-b9d2-5db37f3745e2 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:17.659660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-baf8b2cb-ce84-4010-b9d2-5db37f3745e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:17.659660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:17.659660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:17.659660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:17.659660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:17.659660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:17.659660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:17.659660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:17.659660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:17.659660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:17.659660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:17.659660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:17.659660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 14:18:17.659660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 14:18:17.659660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 14:18:17.659660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 14:18:17.660807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:17.660807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:17.660807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:17.660807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:17.660807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:17.660807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:17.660807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:17.660807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:17.660807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:17.660807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:17.660807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:17.660807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:17.660807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:17.660807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:17.660807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:17.660807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:17.660807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 14:18:17.661795 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 14:18:17.661795 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 14:18:17.661795 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:18:17.661795 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 634/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:17.666096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04208275-5c0f-4ae1-a1f6-aadc451af1eb None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:17.765006 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04208275-5c0f-4ae1-a1f6-aadc451af1eb None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:17.799121 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69d6ec-04b7-426a-b5f3-caf9125bf1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:17.799783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69d6ec-04b7-426a-b5f3-caf9125bf1ac None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:17.800066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69d6ec-04b7-426a-b5f3-caf9125bf1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:17.800343 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69d6ec-04b7-426a-b5f3-caf9125bf1ac None admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:17.801023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69d6ec-04b7-426a-b5f3-caf9125bf1ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:17.801340 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69d6ec-04b7-426a-b5f3-caf9125bf1ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:17.802702 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69d6ec-04b7-426a-b5f3-caf9125bf1ac None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:17.814627 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 650/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 5 14:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:17.820684 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8b2cb-ce84-4010-b9d2-5db37f3745e2 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:17.922250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8b2cb-ce84-4010-b9d2-5db37f3745e2 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:17.935664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:17.955119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f612ae1-b414-4851-a44b-1d804b9f3535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:17.961473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f612ae1-b414-4851-a44b-1d804b9f3535 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:17.961473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f612ae1-b414-4851-a44b-1d804b9f3535 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:17.961473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f612ae1-b414-4851-a44b-1d804b9f3535 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:17.962529 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f612ae1-b414-4851-a44b-1d804b9f3535 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:17.964082 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f612ae1-b414-4851-a44b-1d804b9f3535 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:17.975040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application [None req-3f612ae1-b414-4851-a44b-1d804b9f3535 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:18:17.975040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:17.975040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:17.975040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:17.975040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:17.975040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:17.975040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:17.975040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:17.975040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:17.975040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:17.975040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:17.975040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:17.975040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 14:18:17.975040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 14:18:17.975040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 14:18:17.976101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 14:18:17.976101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:17.976101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:17.976101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:17.976101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:17.976101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:17.976101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:17.976101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:17.976101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:17.976101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:17.976101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:17.976101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:17.976101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:17.976101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:17.976101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:17.976101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:17.976101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:17.977273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:18:17.977273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:18:17.977273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:18:17.977273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:18:17.977273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:18:17.977273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:18:17.977273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 14:18:17.977273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: ERROR keystone.server.flask.application Sep 05 14:18:17.977273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 635/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 5 14:18:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:17.981941 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69d6ec-04b7-426a-b5f3-caf9125bf1ac None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:18.036211 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:18.085198 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69d6ec-04b7-426a-b5f3-caf9125bf1ac None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:18.117028 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb921688-d6d8-408f-8535-5e5433d06db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:18.118757 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb921688-d6d8-408f-8535-5e5433d06db0 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:18.118757 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb921688-d6d8-408f-8535-5e5433d06db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:18.118757 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb921688-d6d8-408f-8535-5e5433d06db0 None admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:18.119047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb921688-d6d8-408f-8535-5e5433d06db0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:18.119181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb921688-d6d8-408f-8535-5e5433d06db0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:18.121091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb921688-d6d8-408f-8535-5e5433d06db0 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:18.132505 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 651/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 5 14:18:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:18.139287 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f612ae1-b414-4851-a44b-1d804b9f3535 None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:18.239605 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f612ae1-b414-4851-a44b-1d804b9f3535 None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:18.277915 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d082d-e766-419c-ac55-4d397117064a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:18.278511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d082d-e766-419c-ac55-4d397117064a None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:18.278808 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d082d-e766-419c-ac55-4d397117064a None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:18.279096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d082d-e766-419c-ac55-4d397117064a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:18.279804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d082d-e766-419c-ac55-4d397117064a None admin] RBAC: Adding query filter params (role.id=affac03b860a4fe0b94e52140e78ab79, scope.project.id=d9ca42deaed344eea726fe6e32356efc, user.id=267825dfb0324be0910203cd5abab377) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:18.280109 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d082d-e766-419c-ac55-4d397117064a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:18.281574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d082d-e766-419c-ac55-4d397117064a None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:18.287260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 636/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 5 14:18:18 2022] GET /identity/v3/role_assignments?user.id=267825dfb0324be0910203cd5abab377&scope.project.id=d9ca42deaed344eea726fe6e32356efc&role.id=affac03b860a4fe0b94e52140e78ab79 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:20.226600 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4336ce9-21d4-444c-9ee7-4cca85d86669 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:20.227176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4336ce9-21d4-444c-9ee7-4cca85d86669 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:20.227176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4336ce9-21d4-444c-9ee7-4cca85d86669 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:20.228528 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 652/1288] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:18:20.233950 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0d29b-0b46-4d0b-be69-289322ca666c None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:20.234396 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0d29b-0b46-4d0b-be69-289322ca666c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:20.234487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0d29b-0b46-4d0b-be69-289322ca666c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:20.252851 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-fbf0d29b-0b46-4d0b-be69-289322ca666c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:18:20.254045 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-fbf0d29b-0b46-4d0b-be69-289322ca666c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:18:20.283784 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:20.318938 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 637/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:18:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:18:20.592924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4336ce9-21d4-444c-9ee7-4cca85d86669 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:20.693247 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4336ce9-21d4-444c-9ee7-4cca85d86669 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:20.705022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:20.727140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9d6fb-bf1e-478b-a5c8-c69377cac590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:20.727766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9d6fb-bf1e-478b-a5c8-c69377cac590 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:20.728180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9d6fb-bf1e-478b-a5c8-c69377cac590 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:20.728343 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9d6fb-bf1e-478b-a5c8-c69377cac590 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:20.728919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9d6fb-bf1e-478b-a5c8-c69377cac590 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:20.730273 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9d6fb-bf1e-478b-a5c8-c69377cac590 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:20.739241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-d1c9d6fb-bf1e-478b-a5c8-c69377cac590 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 14:18:20.739241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:20.739241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:18:20.739241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:18:20.739241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:18:20.739241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:18:20.739241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:18:20.739241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:18:20.739241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 14:18:20.739241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:20.739241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:18:20.739241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:20.739241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:20.739241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:20.739241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:20.739241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:20.739241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:20.740452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:20.740452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:20.740452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:20.740452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:20.740452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:20.740452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:20.740452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:18:20.740452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:18:20.740452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:18:20.740452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:18:20.740452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:20.740452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:20.740452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:20.740452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:20.740452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:20.740452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:20.741344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:20.741344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:20.741344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:20.741344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:20.741344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:20.741344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:20.741344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:20.741344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:20.741344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:20.741344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:20.741344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:18:20.741344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:18:20.741344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 14:18:20.741344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:20.741344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 653/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 5 14:18:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:20.747193 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0d29b-0b46-4d0b-be69-289322ca666c None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:20.843372 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0d29b-0b46-4d0b-be69-289322ca666c None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:20.877225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b370fa3-d547-4a83-a94c-041ced6e877c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:20.877820 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b370fa3-d547-4a83-a94c-041ced6e877c None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:20.878097 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b370fa3-d547-4a83-a94c-041ced6e877c None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:20.878365 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b370fa3-d547-4a83-a94c-041ced6e877c None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:20.879085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b370fa3-d547-4a83-a94c-041ced6e877c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:20.885664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b370fa3-d547-4a83-a94c-041ced6e877c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:20.885664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b370fa3-d547-4a83-a94c-041ced6e877c None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:20.885664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 638/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Sep 5 14:18:20 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:20.891346 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9d6fb-bf1e-478b-a5c8-c69377cac590 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:20.988133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9d6fb-bf1e-478b-a5c8-c69377cac590 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:21.020635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333da420-3150-4070-93cf-59fc24f8fdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:21.021236 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333da420-3150-4070-93cf-59fc24f8fdf6 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:21.021520 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333da420-3150-4070-93cf-59fc24f8fdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:21.021795 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333da420-3150-4070-93cf-59fc24f8fdf6 None admin] PATH_INFO: `/v3/services` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:21.022417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333da420-3150-4070-93cf-59fc24f8fdf6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:21.022717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333da420-3150-4070-93cf-59fc24f8fdf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:21.024104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333da420-3150-4070-93cf-59fc24f8fdf6 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:21.028371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 654/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Sep 5 14:18:20 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:22.916482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a360399-4d4a-431e-a85b-916b30f92bb9 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:22.917052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a360399-4d4a-431e-a85b-916b30f92bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:22.917149 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a360399-4d4a-431e-a85b-916b30f92bb9 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:22.919081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 639/1293] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:18:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:18:22.924547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ba40d-5385-4c0d-99a4-22f60c7dc6e5 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:22.924984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ba40d-5385-4c0d-99a4-22f60c7dc6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:22.925100 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ba40d-5385-4c0d-99a4-22f60c7dc6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:22.943776 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-562ba40d-5385-4c0d-99a4-22f60c7dc6e5 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:18:22.945209 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-562ba40d-5385-4c0d-99a4-22f60c7dc6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:18:22.974626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:23.012841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 655/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:18:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 14:18:23.285019 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a360399-4d4a-431e-a85b-916b30f92bb9 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:23.316511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:23.385581 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a360399-4d4a-431e-a85b-916b30f92bb9 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:23.418587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf3a7f-ccf1-4246-b038-8b2b7abd3b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:23.419214 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf3a7f-ccf1-4246-b038-8b2b7abd3b9c None admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:23.419494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf3a7f-ccf1-4246-b038-8b2b7abd3b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:23.419764 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf3a7f-ccf1-4246-b038-8b2b7abd3b9c None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:23.420398 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf3a7f-ccf1-4246-b038-8b2b7abd3b9c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:23.421759 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf3a7f-ccf1-4246-b038-8b2b7abd3b9c None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:23.552418 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 640/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 5 14:18:23 2022] POST /identity/v3/services => generated 246 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:18:25.460848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b12f8-d7f6-44f1-adf0-a248bb96ffdf None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:25.461397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b12f8-d7f6-44f1-adf0-a248bb96ffdf None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:25.461397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b12f8-d7f6-44f1-adf0-a248bb96ffdf None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:25.462765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 656/1296] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:18:25.468231 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f1b8a-8ea8-42b7-a1c0-c1a9bf0c21b1 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:25.468655 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f1b8a-8ea8-42b7-a1c0-c1a9bf0c21b1 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:25.468776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f1b8a-8ea8-42b7-a1c0-c1a9bf0c21b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:25.487258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-514f1b8a-8ea8-42b7-a1c0-c1a9bf0c21b1 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:18:25.488487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-514f1b8a-8ea8-42b7-a1c0-c1a9bf0c21b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:18:25.560143 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 641/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:18:25 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:18:25.826952 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b12f8-d7f6-44f1-adf0-a248bb96ffdf None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:25.929859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b12f8-d7f6-44f1-adf0-a248bb96ffdf None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:25.966960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6de431-a3c0-47b2-992d-6f0344394eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:25.967588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6de431-a3c0-47b2-992d-6f0344394eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:25.967868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6de431-a3c0-47b2-992d-6f0344394eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:25.968140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6de431-a3c0-47b2-992d-6f0344394eb9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:25.968717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6de431-a3c0-47b2-992d-6f0344394eb9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:25.970138 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6de431-a3c0-47b2-992d-6f0344394eb9 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:25.978742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-bf6de431-a3c0-47b2-992d-6f0344394eb9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 14:18:25.978742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:25.978742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:18:25.978742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:18:25.978742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:18:25.978742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:18:25.978742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:18:25.978742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:18:25.978742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 14:18:25.978742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:25.978742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:18:25.978742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:25.978742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:25.978742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:25.978742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:25.978742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:25.978742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:25.980530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:25.980530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:25.980530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:25.980530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:25.980530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:25.980530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:25.980530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:18:25.980530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:18:25.980530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:18:25.980530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:18:25.980530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:25.980530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:25.980530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:25.980530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:25.980530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:25.980530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:25.981756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:25.981756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:25.981756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:25.981756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:25.981756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:25.981756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:25.981756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:25.981756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:25.981756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:25.981756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:25.981756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:18:25.981756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:18:25.981756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 14:18:25.981756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:25.981756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 657/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 5 14:18:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:25.986469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f1b8a-8ea8-42b7-a1c0-c1a9bf0c21b1 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:26.004789 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:26.084545 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f1b8a-8ea8-42b7-a1c0-c1a9bf0c21b1 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:26.120289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c4164-b4c3-48f2-a134-2499972db7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:26.120891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c4164-b4c3-48f2-a134-2499972db7ee None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:26.121167 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c4164-b4c3-48f2-a134-2499972db7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:26.121441 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c4164-b4c3-48f2-a134-2499972db7ee None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:26.122106 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c4164-b4c3-48f2-a134-2499972db7ee None admin] RBAC: Adding query filter params (name=placement) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:26.122432 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c4164-b4c3-48f2-a134-2499972db7ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:26.123819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c4164-b4c3-48f2-a134-2499972db7ee None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:26.135286 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 642/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Sep 5 14:18:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:26.135511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6de431-a3c0-47b2-992d-6f0344394eb9 None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:26.166579 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:26.232251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6de431-a3c0-47b2-992d-6f0344394eb9 None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:26.265673 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83e8a2-6861-4439-badb-035f66a5ba60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:26.266462 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83e8a2-6861-4439-badb-035f66a5ba60 None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:26.267010 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83e8a2-6861-4439-badb-035f66a5ba60 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:26.267563 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83e8a2-6861-4439-badb-035f66a5ba60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:26.268807 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83e8a2-6861-4439-badb-035f66a5ba60 None admin] RBAC: Adding query filter params (interface=public, service_id=7e3ef1d60df54700843046bc31333394, region_id=RegionOne) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:26.269521 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83e8a2-6861-4439-badb-035f66a5ba60 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:26.276326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83e8a2-6861-4439-badb-035f66a5ba60 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:26.281739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 658/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Sep 5 14:18:26 2022] GET /identity/v3/endpoints?service_id=7e3ef1d60df54700843046bc31333394&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:26.287420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c4164-b4c3-48f2-a134-2499972db7ee None admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:26.367937 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:26.387669 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c4164-b4c3-48f2-a134-2499972db7ee None admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:26.423760 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b7c98-e70e-4e35-bf7c-f5929cb1c922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:26.424388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b7c98-e70e-4e35-bf7c-f5929cb1c922 None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:26.424673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b7c98-e70e-4e35-bf7c-f5929cb1c922 None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:26.424943 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b7c98-e70e-4e35-bf7c-f5929cb1c922 None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:26.425556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b7c98-e70e-4e35-bf7c-f5929cb1c922 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:26.427087 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b7c98-e70e-4e35-bf7c-f5929cb1c922 None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:26.435027 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 643/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 5 14:18:26 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:18:28.350209 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9898a-ee28-4efc-9de8-f14fbc24db57 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:28.350818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9898a-ee28-4efc-9de8-f14fbc24db57 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:28.350818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9898a-ee28-4efc-9de8-f14fbc24db57 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:28.352054 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 659/1302] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:18:28.357737 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247081f-1f40-4d20-98ed-3ef3be3e8359 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:28.358210 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247081f-1f40-4d20-98ed-3ef3be3e8359 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:28.358309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247081f-1f40-4d20-98ed-3ef3be3e8359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:28.376730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-1247081f-1f40-4d20-98ed-3ef3be3e8359 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:18:28.377916 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-1247081f-1f40-4d20-98ed-3ef3be3e8359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:18:28.443290 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 644/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:18:28 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 14:18:28.709869 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9898a-ee28-4efc-9de8-f14fbc24db57 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:28.804020 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:28.810498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9898a-ee28-4efc-9de8-f14fbc24db57 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:28.843275 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85ea20-eff9-4484-aad1-628c5f08c44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:28.843890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85ea20-eff9-4484-aad1-628c5f08c44d None admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:28.844169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85ea20-eff9-4484-aad1-628c5f08c44d None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:28.844441 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85ea20-eff9-4484-aad1-628c5f08c44d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:28.845121 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85ea20-eff9-4484-aad1-628c5f08c44d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:28.846477 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85ea20-eff9-4484-aad1-628c5f08c44d None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:28.855438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-1c85ea20-eff9-4484-aad1-628c5f08c44d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 14:18:28.855438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:28.855438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 14:18:28.855438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 14:18:28.855438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 14:18:28.855438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 14:18:28.855438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 14:18:28.855438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:18:28.855438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 14:18:28.855438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:28.855438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 14:18:28.855438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:28.855438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:18:28.855438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:18:28.855438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:18:28.855438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:18:28.855438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:18:28.856783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:18:28.856783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:18:28.856783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:18:28.856783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:18:28.856783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:18:28.856783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:18:28.856783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 14:18:28.856783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 14:18:28.856783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 14:18:28.856783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 14:18:28.856783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:18:28.856783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:18:28.856783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:18:28.856783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:18:28.856783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:18:28.856783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:18:28.857724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:18:28.857724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:18:28.857724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:18:28.857724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:18:28.857724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:18:28.857724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:18:28.857724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:18:28.857724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:18:28.857724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:18:28.857724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:18:28.857724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 14:18:28.857724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 14:18:28.857724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 14:18:28.857724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:18:28.857724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 660/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 5 14:18:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 14:18:28.862586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247081f-1f40-4d20-98ed-3ef3be3e8359 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:28.936248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:28.958199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247081f-1f40-4d20-98ed-3ef3be3e8359 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:28.991746 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2efec-a630-4dab-94b6-83106327ce2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:28.992344 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2efec-a630-4dab-94b6-83106327ce2c None admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:28.992629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2efec-a630-4dab-94b6-83106327ce2c None admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:28.992897 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2efec-a630-4dab-94b6-83106327ce2c None admin] PATH_INFO: `/v3/services` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:28.993517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2efec-a630-4dab-94b6-83106327ce2c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:18:28.993817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2efec-a630-4dab-94b6-83106327ce2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:28.995207 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2efec-a630-4dab-94b6-83106327ce2c None admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:29.000333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 645/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Sep 5 14:18:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:18:29.005886 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85ea20-eff9-4484-aad1-628c5f08c44d None admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:18:29.106382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85ea20-eff9-4484-aad1-628c5f08c44d None admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:18:29.138970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa58d4-b7d5-4b8c-b3d4-cdfe06228b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:18:29.139636 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa58d4-b7d5-4b8c-b3d4-cdfe06228b08 None admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:18:29.139915 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa58d4-b7d5-4b8c-b3d4-cdfe06228b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:18:29.140184 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa58d4-b7d5-4b8c-b3d4-cdfe06228b08 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:18:29.140842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa58d4-b7d5-4b8c-b3d4-cdfe06228b08 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:18:29.142188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa58d4-b7d5-4b8c-b3d4-cdfe06228b08 None admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:18:29.263357 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 661/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:18:29 2022] POST /identity/v3/endpoints => generated 331 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:18:39.001552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=30 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:18:39.258344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:04.463514 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d47a8-0af0-4adf-97cd-33056b578af4 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:04.464048 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d47a8-0af0-4adf-97cd-33056b578af4 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:04.464048 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d47a8-0af0-4adf-97cd-33056b578af4 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:04.470942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a4635-90da-4fdb-9a06-e65aed8fc0df None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:04.472096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a4635-90da-4fdb-9a06-e65aed8fc0df None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:04.472181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a4635-90da-4fdb-9a06-e65aed8fc0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:04.476851 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 646/1307] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:26:04 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:04.562168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-480a4635-90da-4fdb-9a06-e65aed8fc0df None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:04.563775 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-480a4635-90da-4fdb-9a06-e65aed8fc0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:04.699233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 662/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:26:04.981665 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92165725-c1de-48d2-b38d-b07edab36f6e None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:04.981958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92165725-c1de-48d2-b38d-b07edab36f6e None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:04.982234 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92165725-c1de-48d2-b38d-b07edab36f6e None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:04.983747 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 647/1309] 10.222.0.15 () {54 vars in 965 bytes} [Mon Sep 5 14:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:04.997854 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcf6da-28b9-4014-8fc7-e8a2c894eb34 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:04.998923 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcf6da-28b9-4014-8fc7-e8a2c894eb34 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:04.999736 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcf6da-28b9-4014-8fc7-e8a2c894eb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:05.120535 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-38bcf6da-28b9-4014-8fc7-e8a2c894eb34 None None] MFA Rules not processed for user `5c73ca2785334fc496b5227d3be655c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:05.122831 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-38bcf6da-28b9-4014-8fc7-e8a2c894eb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:05.148523 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:05.219098 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 663/1310] 10.222.0.15 () {58 vars in 1062 bytes} [Mon Sep 5 14:26:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:26:05.229546 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92165725-c1de-48d2-b38d-b07edab36f6e None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:26:05.231367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-92165725-c1de-48d2-b38d-b07edab36f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:05.353625 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92165725-c1de-48d2-b38d-b07edab36f6e None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:26:05.430462 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a626bac-3834-4e57-973f-2cfbc72b0d69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:26:05.431108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a626bac-3834-4e57-973f-2cfbc72b0d69 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:05.431394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a626bac-3834-4e57-973f-2cfbc72b0d69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:05.431670 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a626bac-3834-4e57-973f-2cfbc72b0d69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:05.433281 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-4a626bac-3834-4e57-973f-2cfbc72b0d69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:05.490535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:05.546462 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a626bac-3834-4e57-973f-2cfbc72b0d69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:26:05.547865 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a626bac-3834-4e57-973f-2cfbc72b0d69 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:26:05.566867 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 648/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:26:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:26:07.775635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e60fad-cffb-4d7a-9606-b2610b65880c None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:07.829060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e60fad-cffb-4d7a-9606-b2610b65880c None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:07.829060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e60fad-cffb-4d7a-9606-b2610b65880c None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:07.829060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 664/1312] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:07.829660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bcf68-f796-48d3-97df-77c3e38cfc5a None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:07.829660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bcf68-f796-48d3-97df-77c3e38cfc5a None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:07.829660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bcf68-f796-48d3-97df-77c3e38cfc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:07.868868 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-659bcf68-f796-48d3-97df-77c3e38cfc5a None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:07.870724 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-659bcf68-f796-48d3-97df-77c3e38cfc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:07.981333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 649/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:26:08.264704 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e60fad-cffb-4d7a-9606-b2610b65880c None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:26:08.266230 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-52e60fad-cffb-4d7a-9606-b2610b65880c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:08.369304 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e60fad-cffb-4d7a-9606-b2610b65880c None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:26:08.438798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:08.441679 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae73638-d4a6-4ed4-a5b7-29dbb19c6404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:26:08.442284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae73638-d4a6-4ed4-a5b7-29dbb19c6404 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:08.442566 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae73638-d4a6-4ed4-a5b7-29dbb19c6404 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:08.442863 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae73638-d4a6-4ed4-a5b7-29dbb19c6404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:08.444465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-6ae73638-d4a6-4ed4-a5b7-29dbb19c6404 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:08.557188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae73638-d4a6-4ed4-a5b7-29dbb19c6404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:26:08.558546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae73638-d4a6-4ed4-a5b7-29dbb19c6404 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:26:08.577116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 665/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:26:10.932740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1d623-649c-4a78-afcf-1220c3e8481e None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:10.933338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1d623-649c-4a78-afcf-1220c3e8481e None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:10.933338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1d623-649c-4a78-afcf-1220c3e8481e None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:10.934637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 650/1315] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:10.939644 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cf561-e744-4fd3-873f-92438c7902ad None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:10.940165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cf561-e744-4fd3-873f-92438c7902ad None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:10.940262 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cf561-e744-4fd3-873f-92438c7902ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:11.004070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-b93cf561-e744-4fd3-873f-92438c7902ad None None] MFA Rules not processed for user `b0cca89fab4740169878ca61aabace33`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:11.005300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b93cf561-e744-4fd3-873f-92438c7902ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:11.055943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:11.100402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 666/1316] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:26:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:26:11.384743 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1d623-649c-4a78-afcf-1220c3e8481e None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:26:11.386288 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-94e1d623-649c-4a78-afcf-1220c3e8481e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:11.396544 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:11.488032 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1d623-649c-4a78-afcf-1220c3e8481e None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:26:11.559255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf347ef-00b2-43c0-804b-8c3d051ddff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:26:11.559867 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf347ef-00b2-43c0-804b-8c3d051ddff9 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:11.560151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf347ef-00b2-43c0-804b-8c3d051ddff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:11.560428 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf347ef-00b2-43c0-804b-8c3d051ddff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:11.562016 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-0bf347ef-00b2-43c0-804b-8c3d051ddff9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:11.623060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:11.668978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf347ef-00b2-43c0-804b-8c3d051ddff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:26:11.670363 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf347ef-00b2-43c0-804b-8c3d051ddff9 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:26:11.688476 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 651/1317] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:26:11 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:26:14.361474 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662045d-7639-4452-b546-e0e8e70508bd None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:14.362066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662045d-7639-4452-b546-e0e8e70508bd None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:14.362066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662045d-7639-4452-b546-e0e8e70508bd None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:14.363440 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 667/1318] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:14.368389 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868738ad-ad92-41a6-9daf-ad485616112b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:14.368797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868738ad-ad92-41a6-9daf-ad485616112b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:14.368937 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868738ad-ad92-41a6-9daf-ad485616112b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:14.433746 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-868738ad-ad92-41a6-9daf-ad485616112b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:14.434997 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-868738ad-ad92-41a6-9daf-ad485616112b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:14.529586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 652/1319] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:26:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:26:14.814380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662045d-7639-4452-b546-e0e8e70508bd None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:26:14.816469 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-2662045d-7639-4452-b546-e0e8e70508bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:14.917817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662045d-7639-4452-b546-e0e8e70508bd None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:26:14.988895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd69fe2-691b-471f-bb03-8838acda971d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:26:14.989486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd69fe2-691b-471f-bb03-8838acda971d service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:14.989762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd69fe2-691b-471f-bb03-8838acda971d service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:14.990032 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd69fe2-691b-471f-bb03-8838acda971d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:14.991631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7dd69fe2-691b-471f-bb03-8838acda971d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:14.998747 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:15.099814 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd69fe2-691b-471f-bb03-8838acda971d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:26:15.101248 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd69fe2-691b-471f-bb03-8838acda971d service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:26:15.118700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 668/1320] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:26:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:26:18.126133 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565fd10-08de-4e18-abbf-00b90462e67e None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:18.126672 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565fd10-08de-4e18-abbf-00b90462e67e None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:18.126672 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565fd10-08de-4e18-abbf-00b90462e67e None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:18.128076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 653/1321] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:18.133373 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cb65d-8d66-4a9b-8e50-b2159dd8f3e8 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:18.133802 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cb65d-8d66-4a9b-8e50-b2159dd8f3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:18.134058 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cb65d-8d66-4a9b-8e50-b2159dd8f3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:18.200380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-4a5cb65d-8d66-4a9b-8e50-b2159dd8f3e8 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:18.201856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-4a5cb65d-8d66-4a9b-8e50-b2159dd8f3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:18.262111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:18.301629 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 669/1322] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:26:18.604082 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c9f6c-f8c5-4334-a8f9-d28f006b63fd None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:18.604474 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c9f6c-f8c5-4334-a8f9-d28f006b63fd None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:18.604838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c9f6c-f8c5-4334-a8f9-d28f006b63fd None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:18.606258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 654/1323] 10.222.0.15 () {54 vars in 965 bytes} [Mon Sep 5 14:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:18.612694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44364fd-1128-4912-acf7-f79d154d6319 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:18.613063 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44364fd-1128-4912-acf7-f79d154d6319 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:18.613402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44364fd-1128-4912-acf7-f79d154d6319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:18.693782 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-d44364fd-1128-4912-acf7-f79d154d6319 None None] MFA Rules not processed for user `5c73ca2785334fc496b5227d3be655c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:18.695011 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d44364fd-1128-4912-acf7-f79d154d6319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:18.773100 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 670/1324] 10.222.0.15 () {58 vars in 1062 bytes} [Mon Sep 5 14:26:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:26:18.781908 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c9f6c-f8c5-4334-a8f9-d28f006b63fd None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:26:18.783511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-438c9f6c-f8c5-4334-a8f9-d28f006b63fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:18.804543 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:18.886440 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c9f6c-f8c5-4334-a8f9-d28f006b63fd None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:26:18.958757 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825d0d4-5df0-4bef-919c-630bd5b2b7b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:26:18.959307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825d0d4-5df0-4bef-919c-630bd5b2b7b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:18.959583 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825d0d4-5df0-4bef-919c-630bd5b2b7b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:18.959852 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825d0d4-5df0-4bef-919c-630bd5b2b7b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:18.961435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a825d0d4-5df0-4bef-919c-630bd5b2b7b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:19.031260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:19.069343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825d0d4-5df0-4bef-919c-630bd5b2b7b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:26:19.070802 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825d0d4-5df0-4bef-919c-630bd5b2b7b8 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:26:19.092884 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 655/1325] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:26:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:26:21.268519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b34aba-8c3b-40ba-ac05-c82ecda857db None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:21.269050 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b34aba-8c3b-40ba-ac05-c82ecda857db None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:21.269050 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b34aba-8c3b-40ba-ac05-c82ecda857db None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:21.270418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 671/1326] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:21.275655 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c430b7-018a-43f3-b3c8-6a60374c4897 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:21.276084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c430b7-018a-43f3-b3c8-6a60374c4897 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:21.276195 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c430b7-018a-43f3-b3c8-6a60374c4897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:21.343535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-a3c430b7-018a-43f3-b3c8-6a60374c4897 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:21.350974 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a3c430b7-018a-43f3-b3c8-6a60374c4897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:21.449086 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 656/1327] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:26:21.727481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b34aba-8c3b-40ba-ac05-c82ecda857db None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:26:21.729009 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b5b34aba-8c3b-40ba-ac05-c82ecda857db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:21.789532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:21.833709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b34aba-8c3b-40ba-ac05-c82ecda857db None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:26:21.907498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925976f-89bb-4b8f-92af-f52197ba63bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:26:21.907988 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925976f-89bb-4b8f-92af-f52197ba63bb service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:21.908269 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925976f-89bb-4b8f-92af-f52197ba63bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:21.909170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925976f-89bb-4b8f-92af-f52197ba63bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:21.910772 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-3925976f-89bb-4b8f-92af-f52197ba63bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:22.016142 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:22.018915 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925976f-89bb-4b8f-92af-f52197ba63bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:26:22.020357 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925976f-89bb-4b8f-92af-f52197ba63bb service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:26:22.045034 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 672/1328] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:26:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:26:23.992903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424b55c-33cb-4f62-b583-7c6c3aab869e None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:23.993374 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424b55c-33cb-4f62-b583-7c6c3aab869e None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:23.993492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424b55c-33cb-4f62-b583-7c6c3aab869e None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:23.994802 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 657/1329] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:24.000111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c268eb-ba9d-4bfd-9fa3-87461a080701 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:24.000529 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c268eb-ba9d-4bfd-9fa3-87461a080701 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:24.000671 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c268eb-ba9d-4bfd-9fa3-87461a080701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:24.066134 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-a0c268eb-ba9d-4bfd-9fa3-87461a080701 None None] MFA Rules not processed for user `b0cca89fab4740169878ca61aabace33`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:24.067388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a0c268eb-ba9d-4bfd-9fa3-87461a080701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:24.186666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 673/1330] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:26:23 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:26:24.470602 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424b55c-33cb-4f62-b583-7c6c3aab869e None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:26:24.472169 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9424b55c-33cb-4f62-b583-7c6c3aab869e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:24.503417 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:24.577465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424b55c-33cb-4f62-b583-7c6c3aab869e None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:26:24.655756 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c3b15-d017-46d1-886b-9e10cde04083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:26:24.656370 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c3b15-d017-46d1-886b-9e10cde04083 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:24.656653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c3b15-d017-46d1-886b-9e10cde04083 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:24.656929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c3b15-d017-46d1-886b-9e10cde04083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:24.658621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-484c3b15-d017-46d1-886b-9e10cde04083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:24.740426 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:24.775342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c3b15-d017-46d1-886b-9e10cde04083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:26:24.776942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c3b15-d017-46d1-886b-9e10cde04083 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:26:24.797914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 658/1331] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:26:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:26:27.209817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdf44e-761e-4777-87a9-240f0a9847c8 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:27.210345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdf44e-761e-4777-87a9-240f0a9847c8 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:27.210345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdf44e-761e-4777-87a9-240f0a9847c8 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:27.211742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 674/1332] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:27.217002 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dad361-157b-4677-acd4-0bbd3af19a8b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:27.217515 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dad361-157b-4677-acd4-0bbd3af19a8b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:27.217614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dad361-157b-4677-acd4-0bbd3af19a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:27.282089 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-59dad361-157b-4677-acd4-0bbd3af19a8b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:27.283325 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-59dad361-157b-4677-acd4-0bbd3af19a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:27.379917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 659/1333] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:26:27.658067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bdf44e-761e-4777-87a9-240f0a9847c8 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:26:27.660155 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-42bdf44e-761e-4777-87a9-240f0a9847c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:27.729922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:27.763231 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bdf44e-761e-4777-87a9-240f0a9847c8 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:26:27.835465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7f2e0-1057-402e-adcf-36ee74f124ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:26:27.836064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7f2e0-1057-402e-adcf-36ee74f124ff service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:27.836348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7f2e0-1057-402e-adcf-36ee74f124ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:27.836622 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7f2e0-1057-402e-adcf-36ee74f124ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:27.838207 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5dd7f2e0-1057-402e-adcf-36ee74f124ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:27.945753 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7f2e0-1057-402e-adcf-36ee74f124ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:26:27.947233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7f2e0-1057-402e-adcf-36ee74f124ff service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:26:27.957839 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:27.965767 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 675/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:26:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:26:29.905194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a12b26-d2a3-4431-b7b5-ca79d3000c09 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:29.905732 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a12b26-d2a3-4431-b7b5-ca79d3000c09 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:29.905732 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a12b26-d2a3-4431-b7b5-ca79d3000c09 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:29.907131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 660/1335] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:29.912192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff3f69-5ba1-4b1a-a9c6-210cb5a00a51 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:29.912577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff3f69-5ba1-4b1a-a9c6-210cb5a00a51 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:29.912735 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff3f69-5ba1-4b1a-a9c6-210cb5a00a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:29.978072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-45ff3f69-5ba1-4b1a-a9c6-210cb5a00a51 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:29.979313 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-45ff3f69-5ba1-4b1a-a9c6-210cb5a00a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:30.073916 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 676/1336] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:26:30.360452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a12b26-d2a3-4431-b7b5-ca79d3000c09 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:26:30.362499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-55a12b26-d2a3-4431-b7b5-ca79d3000c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:30.401757 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:30.463615 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a12b26-d2a3-4431-b7b5-ca79d3000c09 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:26:30.535823 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac5f39-fdfb-4371-aeb2-e6d6f4e8b72a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:26:30.536426 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac5f39-fdfb-4371-aeb2-e6d6f4e8b72a service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:30.536710 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac5f39-fdfb-4371-aeb2-e6d6f4e8b72a service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:30.536984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac5f39-fdfb-4371-aeb2-e6d6f4e8b72a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:30.538580 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d9ac5f39-fdfb-4371-aeb2-e6d6f4e8b72a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:30.620310 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:30.645051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac5f39-fdfb-4371-aeb2-e6d6f4e8b72a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:26:30.646499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac5f39-fdfb-4371-aeb2-e6d6f4e8b72a service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:26:30.664740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 661/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:26:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:26:33.143751 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5dc8bf-0590-47d7-b3d2-c36859bd0289 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:33.144252 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5dc8bf-0590-47d7-b3d2-c36859bd0289 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:33.144372 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5dc8bf-0590-47d7-b3d2-c36859bd0289 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:33.145625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 677/1338] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:33.150599 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d288cce-fb30-487c-8318-0ee1cb2ec6cd None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:33.151043 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d288cce-fb30-487c-8318-0ee1cb2ec6cd None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:33.151177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d288cce-fb30-487c-8318-0ee1cb2ec6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:33.216603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-1d288cce-fb30-487c-8318-0ee1cb2ec6cd None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:33.217793 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-1d288cce-fb30-487c-8318-0ee1cb2ec6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:33.311987 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 662/1339] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:26:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:26:33.591669 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5dc8bf-0590-47d7-b3d2-c36859bd0289 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:26:33.593260 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-9e5dc8bf-0590-47d7-b3d2-c36859bd0289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:33.669641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:33.697300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5dc8bf-0590-47d7-b3d2-c36859bd0289 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:26:33.769585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb564b-3898-4dc8-a288-0ffd60b22fd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:26:33.770185 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb564b-3898-4dc8-a288-0ffd60b22fd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:33.770465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb564b-3898-4dc8-a288-0ffd60b22fd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:33.770739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb564b-3898-4dc8-a288-0ffd60b22fd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:33.772358 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b6eb564b-3898-4dc8-a288-0ffd60b22fd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:33.879444 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb564b-3898-4dc8-a288-0ffd60b22fd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:26:33.880798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb564b-3898-4dc8-a288-0ffd60b22fd8 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:26:33.898716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 678/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:26:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:26:37.886391 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fca374-8bc2-4827-a566-8987946ff961 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:37.886948 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fca374-8bc2-4827-a566-8987946ff961 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:37.886948 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fca374-8bc2-4827-a566-8987946ff961 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:37.888309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 663/1341] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:37.893486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6d06a-1504-4423-ac06-e21f7423e1ad None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:37.893925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6d06a-1504-4423-ac06-e21f7423e1ad None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:37.894021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6d06a-1504-4423-ac06-e21f7423e1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:37.899560 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:37.964426 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-6fd6d06a-1504-4423-ac06-e21f7423e1ad None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:37.965672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-6fd6d06a-1504-4423-ac06-e21f7423e1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:38.065112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 679/1342] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:26:38.345411 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fca374-8bc2-4827-a566-8987946ff961 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:26:38.346972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e6fca374-8bc2-4827-a566-8987946ff961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:38.397183 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:38.456893 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fca374-8bc2-4827-a566-8987946ff961 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:26:38.530380 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2eeb3-263d-4b0c-b054-9f3341bdfacc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:26:38.531018 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2eeb3-263d-4b0c-b054-9f3341bdfacc service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:38.531300 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2eeb3-263d-4b0c-b054-9f3341bdfacc service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:38.531574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2eeb3-263d-4b0c-b054-9f3341bdfacc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:38.533307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-45b2eeb3-263d-4b0c-b054-9f3341bdfacc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:38.627934 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:38.644241 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2eeb3-263d-4b0c-b054-9f3341bdfacc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:26:38.645689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2eeb3-263d-4b0c-b054-9f3341bdfacc service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:26:38.664242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 664/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:26:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:26:41.756975 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06f555-7375-4c59-a411-2a732276ee66 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:41.757536 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06f555-7375-4c59-a411-2a732276ee66 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:41.757536 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06f555-7375-4c59-a411-2a732276ee66 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:41.758897 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 680/1344] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:41.763957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b539b7-bffa-4f26-b31b-f1dfef9d5087 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:41.764410 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b539b7-bffa-4f26-b31b-f1dfef9d5087 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:41.764524 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b539b7-bffa-4f26-b31b-f1dfef9d5087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:41.830733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-60b539b7-bffa-4f26-b31b-f1dfef9d5087 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:41.831978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-60b539b7-bffa-4f26-b31b-f1dfef9d5087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:41.927261 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 665/1345] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:26:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:26:42.210345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06f555-7375-4c59-a411-2a732276ee66 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:26:42.211942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5a06f555-7375-4c59-a411-2a732276ee66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:42.303040 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:42.320642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06f555-7375-4c59-a411-2a732276ee66 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:26:42.400555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac063431-6ca1-49bd-8ab7-08710ca73dda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:26:42.401084 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac063431-6ca1-49bd-8ab7-08710ca73dda service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:42.401366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac063431-6ca1-49bd-8ab7-08710ca73dda service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:42.401655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac063431-6ca1-49bd-8ab7-08710ca73dda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:42.403339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ac063431-6ca1-49bd-8ab7-08710ca73dda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:42.515180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac063431-6ca1-49bd-8ab7-08710ca73dda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:26:42.516629 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac063431-6ca1-49bd-8ab7-08710ca73dda service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:26:42.535650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 681/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:26:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:26:46.617684 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2f23d-19ec-493b-adfc-11955bdeb408 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:46.618236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2f23d-19ec-493b-adfc-11955bdeb408 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:46.618236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2f23d-19ec-493b-adfc-11955bdeb408 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:46.619760 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 666/1347] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:46.624944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be88fc4-1aae-457a-8761-49bfb70ff8e2 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:46.625428 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be88fc4-1aae-457a-8761-49bfb70ff8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:46.625526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be88fc4-1aae-457a-8761-49bfb70ff8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:46.641319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:46.696330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-2be88fc4-1aae-457a-8761-49bfb70ff8e2 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:46.697569 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-2be88fc4-1aae-457a-8761-49bfb70ff8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:46.797726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 682/1348] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:26:47.083245 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2f23d-19ec-493b-adfc-11955bdeb408 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:26:47.084736 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-5ce2f23d-19ec-493b-adfc-11955bdeb408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:47.148306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:47.198138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2f23d-19ec-493b-adfc-11955bdeb408 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:26:47.272294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56ba59-0698-4691-9ffc-3c556d33c293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:26:47.272894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56ba59-0698-4691-9ffc-3c556d33c293 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:47.273172 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56ba59-0698-4691-9ffc-3c556d33c293 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:47.273440 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56ba59-0698-4691-9ffc-3c556d33c293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:47.275058 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-6d56ba59-0698-4691-9ffc-3c556d33c293 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:47.379828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:47.383384 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56ba59-0698-4691-9ffc-3c556d33c293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:26:47.385280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56ba59-0698-4691-9ffc-3c556d33c293 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:26:47.410704 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 667/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:26:52.321399 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7831768-2dfa-456e-8ba2-2354606270bb None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:52.321832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7831768-2dfa-456e-8ba2-2354606270bb None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:52.321943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7831768-2dfa-456e-8ba2-2354606270bb None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:52.324868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 683/1350] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 5 14:26:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:52.334186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161927f8-2f20-4efe-9cd0-7b32a061c7fe None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:52.334186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161927f8-2f20-4efe-9cd0-7b32a061c7fe None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:52.334186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161927f8-2f20-4efe-9cd0-7b32a061c7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:52.413235 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-161927f8-2f20-4efe-9cd0-7b32a061c7fe None None] MFA Rules not processed for user `267825dfb0324be0910203cd5abab377`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:52.414533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-161927f8-2f20-4efe-9cd0-7b32a061c7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:52.579164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 668/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 5 14:26:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:26:56.649893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a9001-2870-4ccd-b29b-360700016550 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:56.650460 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a9001-2870-4ccd-b29b-360700016550 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:56.650460 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a9001-2870-4ccd-b29b-360700016550 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:56.652083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 684/1352] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 5 14:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:56.657603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe030c8e-a6ec-4e1c-b8b2-ba3b96f0160c None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:56.658506 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe030c8e-a6ec-4e1c-b8b2-ba3b96f0160c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:56.658741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe030c8e-a6ec-4e1c-b8b2-ba3b96f0160c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:56.713881 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:56.739218 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-fe030c8e-a6ec-4e1c-b8b2-ba3b96f0160c None None] MFA Rules not processed for user `267825dfb0324be0910203cd5abab377`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:56.741049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-fe030c8e-a6ec-4e1c-b8b2-ba3b96f0160c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:56.796067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=59 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:56.852188 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 669/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 5 14:26:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:26:58.213725 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e218ed-3224-426c-bab3-b65b8a06dd0f None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:58.214226 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e218ed-3224-426c-bab3-b65b8a06dd0f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:58.214226 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e218ed-3224-426c-bab3-b65b8a06dd0f None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:58.215778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 685/1354] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 5 14:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:26:58.221398 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f8503-b7bb-4ef6-8f73-69f04745ea2d None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:26:58.221806 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f8503-b7bb-4ef6-8f73-69f04745ea2d None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:26:58.221943 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f8503-b7bb-4ef6-8f73-69f04745ea2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:26:58.301778 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-7d6f8503-b7bb-4ef6-8f73-69f04745ea2d None None] MFA Rules not processed for user `267825dfb0324be0910203cd5abab377`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:26:58.303039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-7d6f8503-b7bb-4ef6-8f73-69f04745ea2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:26:58.347902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:26:58.411643 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 670/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 5 14:26:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:27:01.764159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafc2c8-0613-44f1-aeae-bcfb2d45f4a5 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:01.764692 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafc2c8-0613-44f1-aeae-bcfb2d45f4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:01.764692 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafc2c8-0613-44f1-aeae-bcfb2d45f4a5 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:01.766214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 686/1356] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:27:01.773382 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dde9b8-69dc-43a6-940d-65ce02bb516a None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:01.773818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dde9b8-69dc-43a6-940d-65ce02bb516a None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:01.773920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dde9b8-69dc-43a6-940d-65ce02bb516a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:01.858590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-a0dde9b8-69dc-43a6-940d-65ce02bb516a None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:01.860014 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a0dde9b8-69dc-43a6-940d-65ce02bb516a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:01.986117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 671/1357] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:02.288579 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa90a2a-1a5c-43a8-985d-ce1b56c790f8 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:02.288867 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa90a2a-1a5c-43a8-985d-ce1b56c790f8 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:02.289323 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa90a2a-1a5c-43a8-985d-ce1b56c790f8 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:02.290671 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 687/1358] 10.222.0.15 () {54 vars in 962 bytes} [Mon Sep 5 14:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:02.299398 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63feeab5-e576-4173-a95b-85914c3b6815 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:02.300208 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63feeab5-e576-4173-a95b-85914c3b6815 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:02.300736 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63feeab5-e576-4173-a95b-85914c3b6815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:02.325952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:02.389677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-63feeab5-e576-4173-a95b-85914c3b6815 None None] MFA Rules not processed for user `709b9ae82dcd47408fc9a6d54a54428a`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:02.390959 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-63feeab5-e576-4173-a95b-85914c3b6815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:02.509220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 672/1359] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Sep 5 14:27:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:02.518301 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa90a2a-1a5c-43a8-985d-ce1b56c790f8 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:02.520095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8fa90a2a-1a5c-43a8-985d-ce1b56c790f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:02.684215 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa90a2a-1a5c-43a8-985d-ce1b56c790f8 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:02.762492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:02.805105 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449dbee-52ab-44ea-9957-bfe52ad3d895 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:02.806563 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449dbee-52ab-44ea-9957-bfe52ad3d895 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:02.811713 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449dbee-52ab-44ea-9957-bfe52ad3d895 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:02.811713 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449dbee-52ab-44ea-9957-bfe52ad3d895 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:02.811713 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d449dbee-52ab-44ea-9957-bfe52ad3d895 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:02.904777 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449dbee-52ab-44ea-9957-bfe52ad3d895 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:27:02.906390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449dbee-52ab-44ea-9957-bfe52ad3d895 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:27:02.926391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 688/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:27:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:27:05.056889 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418846f-2209-42e4-bb10-2e1c21686e5b None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:05.057421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418846f-2209-42e4-bb10-2e1c21686e5b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:05.057519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418846f-2209-42e4-bb10-2e1c21686e5b None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:05.059184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 673/1361] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:05.065432 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b056a90-ec7e-4fe2-85f2-703e77cf640e None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:05.066030 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b056a90-ec7e-4fe2-85f2-703e77cf640e None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:05.066327 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b056a90-ec7e-4fe2-85f2-703e77cf640e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:05.142114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-3b056a90-ec7e-4fe2-85f2-703e77cf640e None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:05.143944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-3b056a90-ec7e-4fe2-85f2-703e77cf640e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:05.182909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:05.247077 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 689/1362] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:05.557933 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418846f-2209-42e4-bb10-2e1c21686e5b None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:05.557933 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b418846f-2209-42e4-bb10-2e1c21686e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:05.645316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:05.730740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418846f-2209-42e4-bb10-2e1c21686e5b None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:05.847568 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae67bc7-d117-41db-bc27-d7a4e6898cf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:05.848083 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae67bc7-d117-41db-bc27-d7a4e6898cf1 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:05.848362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae67bc7-d117-41db-bc27-d7a4e6898cf1 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:05.848631 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae67bc7-d117-41db-bc27-d7a4e6898cf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:05.850223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-cae67bc7-d117-41db-bc27-d7a4e6898cf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:05.873872 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:05.948874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae67bc7-d117-41db-bc27-d7a4e6898cf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:27:05.950501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae67bc7-d117-41db-bc27-d7a4e6898cf1 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:27:05.972718 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 674/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:27:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:27:08.410500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae57665-93e5-4f0d-bbb2-3c53c1aaffd1 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:08.411116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae57665-93e5-4f0d-bbb2-3c53c1aaffd1 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:08.411116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae57665-93e5-4f0d-bbb2-3c53c1aaffd1 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:08.412417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 690/1364] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:08.417766 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659653e-f0ce-4072-a4eb-689f9e4b5b65 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:08.418180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659653e-f0ce-4072-a4eb-689f9e4b5b65 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:08.418400 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659653e-f0ce-4072-a4eb-689f9e4b5b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:08.485649 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-8659653e-f0ce-4072-a4eb-689f9e4b5b65 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:08.486914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8659653e-f0ce-4072-a4eb-689f9e4b5b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:08.596193 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:08.602837 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 675/1365] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:08.895308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ad4e8-2aa1-4dde-9f24-38c5f8d0bd00 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:08.895619 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ad4e8-2aa1-4dde-9f24-38c5f8d0bd00 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:08.895872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ad4e8-2aa1-4dde-9f24-38c5f8d0bd00 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:08.897534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 691/1366] 10.222.0.15 () {54 vars in 962 bytes} [Mon Sep 5 14:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:08.903109 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65062a-22eb-4d89-af09-83fb7f8f79e6 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:08.903518 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65062a-22eb-4d89-af09-83fb7f8f79e6 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:08.903866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65062a-22eb-4d89-af09-83fb7f8f79e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:08.983680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-9c65062a-22eb-4d89-af09-83fb7f8f79e6 None None] MFA Rules not processed for user `709b9ae82dcd47408fc9a6d54a54428a`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:08.984880 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9c65062a-22eb-4d89-af09-83fb7f8f79e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:09.119024 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 676/1367] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Sep 5 14:27:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:09.127396 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ad4e8-2aa1-4dde-9f24-38c5f8d0bd00 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:09.129085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-4f5ad4e8-2aa1-4dde-9f24-38c5f8d0bd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:09.295581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ad4e8-2aa1-4dde-9f24-38c5f8d0bd00 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:09.316610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:09.442568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769da89-27ac-422b-b584-d26159bdf47e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:09.442568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769da89-27ac-422b-b584-d26159bdf47e service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:09.442568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769da89-27ac-422b-b584-d26159bdf47e service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:09.442568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769da89-27ac-422b-b584-d26159bdf47e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:09.443453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0769da89-27ac-422b-b584-d26159bdf47e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:09.527776 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769da89-27ac-422b-b584-d26159bdf47e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:27:09.529580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769da89-27ac-422b-b584-d26159bdf47e service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:27:09.548151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 692/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:27:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:27:09.991332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202855a-678c-4210-bc97-2b92b5983464 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:09.991889 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202855a-678c-4210-bc97-2b92b5983464 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:09.991889 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202855a-678c-4210-bc97-2b92b5983464 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:09.997066 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 677/1369] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:09.999202 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cccff-75a6-4595-bcb1-03cafb342176 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:09.999507 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cccff-75a6-4595-bcb1-03cafb342176 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:09.999788 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cccff-75a6-4595-bcb1-03cafb342176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:10.033343 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:10.070404 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-4b9cccff-75a6-4595-bcb1-03cafb342176 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:10.072096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-4b9cccff-75a6-4595-bcb1-03cafb342176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:10.178844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 693/1370] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:10.759954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bc675-32bf-415e-9a48-eb685141d4a0 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:10.760246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bc675-32bf-415e-9a48-eb685141d4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:10.760523 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bc675-32bf-415e-9a48-eb685141d4a0 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:10.761997 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 678/1371] 10.222.0.15 () {54 vars in 964 bytes} [Mon Sep 5 14:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:10.766888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433df20-a557-4b4b-93f1-9b6548fbeb36 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:10.767416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433df20-a557-4b4b-93f1-9b6548fbeb36 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:10.767584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433df20-a557-4b4b-93f1-9b6548fbeb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:10.845400 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-4433df20-a557-4b4b-93f1-9b6548fbeb36 None None] MFA Rules not processed for user `e49ac59302cb42f7942954d3859d3675`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:10.846440 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-4433df20-a557-4b4b-93f1-9b6548fbeb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:10.898894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:10.928797 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 694/1372] 10.222.0.15 () {58 vars in 1061 bytes} [Mon Sep 5 14:27:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:10.936447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bc675-32bf-415e-9a48-eb685141d4a0 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:10.937936 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-fa6bc675-32bf-415e-9a48-eb685141d4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:10.990206 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:11.044517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bc675-32bf-415e-9a48-eb685141d4a0 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:11.121268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06eca5-8746-4e3e-8833-86751ca57200 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:11.121880 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06eca5-8746-4e3e-8833-86751ca57200 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:11.122385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06eca5-8746-4e3e-8833-86751ca57200 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:11.122822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06eca5-8746-4e3e-8833-86751ca57200 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:11.124288 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ba06eca5-8746-4e3e-8833-86751ca57200 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:11.225986 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:11.240300 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06eca5-8746-4e3e-8833-86751ca57200 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:27:11.241772 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06eca5-8746-4e3e-8833-86751ca57200 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:27:11.263564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 679/1373] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:27:11.702703 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b4cca-5e10-4ce1-8661-d7d935aa1af2 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:11.703035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b4cca-5e10-4ce1-8661-d7d935aa1af2 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:11.703305 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b4cca-5e10-4ce1-8661-d7d935aa1af2 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:11.704961 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 695/1374] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:11.710022 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29e873-26d2-4285-8b35-e8d6d39a87c1 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:11.710315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29e873-26d2-4285-8b35-e8d6d39a87c1 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:11.710593 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29e873-26d2-4285-8b35-e8d6d39a87c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:11.777234 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-0b29e873-26d2-4285-8b35-e8d6d39a87c1 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:11.778678 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-0b29e873-26d2-4285-8b35-e8d6d39a87c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:11.874619 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 680/1375] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:12.165436 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b4cca-5e10-4ce1-8661-d7d935aa1af2 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:12.167029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-292b4cca-5e10-4ce1-8661-d7d935aa1af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:12.314862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b4cca-5e10-4ce1-8661-d7d935aa1af2 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:12.372925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:12.435660 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3074877-4023-4575-a684-460e9e3ba542 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:12.436272 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3074877-4023-4575-a684-460e9e3ba542 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:12.436554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3074877-4023-4575-a684-460e9e3ba542 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:12.436854 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3074877-4023-4575-a684-460e9e3ba542 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:12.438522 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e3074877-4023-4575-a684-460e9e3ba542 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:12.529834 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3074877-4023-4575-a684-460e9e3ba542 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:27:12.531465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3074877-4023-4575-a684-460e9e3ba542 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:27:12.549256 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 696/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:27:14.607128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376053ba-6349-4fd3-9593-9f77958cffc9 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:14.607676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376053ba-6349-4fd3-9593-9f77958cffc9 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:14.607676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376053ba-6349-4fd3-9593-9f77958cffc9 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:14.609085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 681/1377] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:14.614088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c286547-961b-4b14-97ba-0468194af210 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:14.614516 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c286547-961b-4b14-97ba-0468194af210 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:14.614616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c286547-961b-4b14-97ba-0468194af210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:14.679043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-3c286547-961b-4b14-97ba-0468194af210 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:14.680232 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-3c286547-961b-4b14-97ba-0468194af210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:14.686769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:14.776481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 697/1378] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:15.075840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376053ba-6349-4fd3-9593-9f77958cffc9 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:15.077819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-376053ba-6349-4fd3-9593-9f77958cffc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:15.148352 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:15.238333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376053ba-6349-4fd3-9593-9f77958cffc9 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:15.362617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00280d7b-790c-46be-9f2e-f9de9a476cdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:15.363279 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00280d7b-790c-46be-9f2e-f9de9a476cdd service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:15.363679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00280d7b-790c-46be-9f2e-f9de9a476cdd service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:15.363953 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00280d7b-790c-46be-9f2e-f9de9a476cdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:15.365613 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-00280d7b-790c-46be-9f2e-f9de9a476cdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:15.387030 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:15.464467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00280d7b-790c-46be-9f2e-f9de9a476cdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:27:15.466130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00280d7b-790c-46be-9f2e-f9de9a476cdd service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:27:15.487661 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 682/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:27:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:27:17.578199 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eeef1d-c28c-4b4f-9ca9-10e5e1c151be None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:17.578751 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eeef1d-c28c-4b4f-9ca9-10e5e1c151be None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:17.578751 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eeef1d-c28c-4b4f-9ca9-10e5e1c151be None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:17.581920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 698/1380] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:27:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:17.586266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96b8ae-7027-49a5-9075-eb1177bc34aa None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:17.586684 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96b8ae-7027-49a5-9075-eb1177bc34aa None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:17.586904 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96b8ae-7027-49a5-9075-eb1177bc34aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:17.658893 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-2f96b8ae-7027-49a5-9075-eb1177bc34aa None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:17.660096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-2f96b8ae-7027-49a5-9075-eb1177bc34aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:17.769025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:17.776221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 683/1381] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:18.067590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eeef1d-c28c-4b4f-9ca9-10e5e1c151be None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:18.069626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-58eeef1d-c28c-4b4f-9ca9-10e5e1c151be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:18.231839 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:18.242770 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eeef1d-c28c-4b4f-9ca9-10e5e1c151be None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:18.363908 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28523fb2-ba9e-4665-9a1f-37ce0a9c26d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:18.364518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28523fb2-ba9e-4665-9a1f-37ce0a9c26d2 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:18.364796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28523fb2-ba9e-4665-9a1f-37ce0a9c26d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:18.365062 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28523fb2-ba9e-4665-9a1f-37ce0a9c26d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:18.366639 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-28523fb2-ba9e-4665-9a1f-37ce0a9c26d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:18.462194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28523fb2-ba9e-4665-9a1f-37ce0a9c26d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:27:18.463939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28523fb2-ba9e-4665-9a1f-37ce0a9c26d2 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:27:18.474648 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:18.482880 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 699/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:27:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:27:20.920034 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df382a-97ee-42db-8e2b-4546ce4551a9 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:20.920502 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df382a-97ee-42db-8e2b-4546ce4551a9 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:20.920600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df382a-97ee-42db-8e2b-4546ce4551a9 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:20.921948 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 684/1383] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:20.927660 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736d8b-d1ba-4931-9457-014b6ab42d42 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:20.928053 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736d8b-d1ba-4931-9457-014b6ab42d42 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:20.928263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736d8b-d1ba-4931-9457-014b6ab42d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:20.999929 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-d8736d8b-d1ba-4931-9457-014b6ab42d42 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:21.001136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d8736d8b-d1ba-4931-9457-014b6ab42d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:21.103595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 700/1384] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:21.391600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df382a-97ee-42db-8e2b-4546ce4551a9 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:21.393119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-80df382a-97ee-42db-8e2b-4546ce4551a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:21.544612 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df382a-97ee-42db-8e2b-4546ce4551a9 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:21.614466 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:21.662708 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607009e-c215-4277-a2e1-965f7be62c4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:21.663265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607009e-c215-4277-a2e1-965f7be62c4f service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:21.663549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607009e-c215-4277-a2e1-965f7be62c4f service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:21.663824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607009e-c215-4277-a2e1-965f7be62c4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:21.665406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d607009e-c215-4277-a2e1-965f7be62c4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:21.759167 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607009e-c215-4277-a2e1-965f7be62c4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:27:21.761578 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607009e-c215-4277-a2e1-965f7be62c4f service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:27:21.785239 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 685/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:27:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:27:24.221065 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85107b0b-439f-43ca-b23b-c1c20320eb10 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:24.221616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85107b0b-439f-43ca-b23b-c1c20320eb10 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:24.221616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85107b0b-439f-43ca-b23b-c1c20320eb10 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:24.222991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 701/1386] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:27:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:27:24.228651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227c35c-fef6-4ddd-b13b-152271defc47 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:24.229064 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227c35c-fef6-4ddd-b13b-152271defc47 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:24.229338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227c35c-fef6-4ddd-b13b-152271defc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:24.308210 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-e227c35c-fef6-4ddd-b13b-152271defc47 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:24.309489 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e227c35c-fef6-4ddd-b13b-152271defc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:24.343959 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:24.559575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 686/1387] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:24.859039 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85107b0b-439f-43ca-b23b-c1c20320eb10 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:24.860629 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-85107b0b-439f-43ca-b23b-c1c20320eb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:24.956583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:25.034160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85107b0b-439f-43ca-b23b-c1c20320eb10 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:25.164515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccbc2c-4265-4070-a7af-c334de6c6172 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:25.165130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccbc2c-4265-4070-a7af-c334de6c6172 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:25.165409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccbc2c-4265-4070-a7af-c334de6c6172 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:25.165680 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccbc2c-4265-4070-a7af-c334de6c6172 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:25.167371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-72ccbc2c-4265-4070-a7af-c334de6c6172 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:25.208339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:25.259948 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccbc2c-4265-4070-a7af-c334de6c6172 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:27:25.261541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccbc2c-4265-4070-a7af-c334de6c6172 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:27:25.279870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 702/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:27:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:27:26.709262 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352e47d-c4df-498c-94c7-743ba592d3d8 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:26.709822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352e47d-c4df-498c-94c7-743ba592d3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:26.709822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352e47d-c4df-498c-94c7-743ba592d3d8 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:26.711195 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 687/1389] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:26.716549 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2fdec-84ec-4ce8-98bb-a730f1fe23f7 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:26.716957 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2fdec-84ec-4ce8-98bb-a730f1fe23f7 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:26.717104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2fdec-84ec-4ce8-98bb-a730f1fe23f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:26.783803 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-c4f2fdec-84ec-4ce8-98bb-a730f1fe23f7 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:26.785022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c4f2fdec-84ec-4ce8-98bb-a730f1fe23f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:26.882781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 703/1390] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:27.177874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9a569-0a41-4680-b9b2-8744f0c9dc71 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:27.178158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9a569-0a41-4680-b9b2-8744f0c9dc71 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:27.178429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9a569-0a41-4680-b9b2-8744f0c9dc71 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:27.179774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 688/1391] 10.222.0.15 () {54 vars in 961 bytes} [Mon Sep 5 14:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:27.184988 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57020ae-c180-4c03-9223-b4b7c4917d6d None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:27.185279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57020ae-c180-4c03-9223-b4b7c4917d6d None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:27.185552 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57020ae-c180-4c03-9223-b4b7c4917d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:27.201592 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:27.262763 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-b57020ae-c180-4c03-9223-b4b7c4917d6d None None] MFA Rules not processed for user `735986478c354ed196c601f12f3ac377`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:27.263991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b57020ae-c180-4c03-9223-b4b7c4917d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:27.343865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 704/1392] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Sep 5 14:27:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:27.352107 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9a569-0a41-4680-b9b2-8744f0c9dc71 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:27.353593 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-4af9a569-0a41-4680-b9b2-8744f0c9dc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:27.369674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e1989-7dfa-4886-a271-2afa3c242c46 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:27.369964 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e1989-7dfa-4886-a271-2afa3c242c46 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:27.370249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e1989-7dfa-4886-a271-2afa3c242c46 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:27.371619 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 705/1393] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 14:27:27.376655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8afb20-b9c2-4632-a8d2-565487723528 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:27.376945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8afb20-b9c2-4632-a8d2-565487723528 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:27.377220 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8afb20-b9c2-4632-a8d2-565487723528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:27.446958 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-8f8afb20-b9c2-4632-a8d2-565487723528 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:27.448114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8f8afb20-b9c2-4632-a8d2-565487723528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:27.457342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9a569-0a41-4680-b9b2-8744f0c9dc71 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:27.505461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:27.515008 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:27.529874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0afb8e-16e0-4a14-8fa3-ee64dda830a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '735986478c354ed196c601f12f3ac377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:27.530478 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0afb8e-16e0-4a14-8fa3-ee64dda830a9 service glance] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:27.530759 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0afb8e-16e0-4a14-8fa3-ee64dda830a9 service glance] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:27.531059 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0afb8e-16e0-4a14-8fa3-ee64dda830a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:27.533200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-bf0afb8e-16e0-4a14-8fa3-ee64dda830a9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:27.547538 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 706/1394] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:27:27.643041 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0afb8e-16e0-4a14-8fa3-ee64dda830a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:27:27.644500 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0afb8e-16e0-4a14-8fa3-ee64dda830a9 service glance] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:27:27.662351 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 689/1395] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:27:27.837256 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8afb20-b9c2-4632-a8d2-565487723528 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:27.838466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8f8afb20-b9c2-4632-a8d2-565487723528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:27.960250 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8afb20-b9c2-4632-a8d2-565487723528 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:28.030965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:28.078915 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3b165-5749-45cd-885e-bc4fab55f3ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:28.080280 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3b165-5749-45cd-885e-bc4fab55f3ef service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:28.080593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3b165-5749-45cd-885e-bc4fab55f3ef service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:28.080865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3b165-5749-45cd-885e-bc4fab55f3ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:28.082518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-37a3b165-5749-45cd-885e-bc4fab55f3ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:28.190232 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3b165-5749-45cd-885e-bc4fab55f3ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:27:28.191897 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3b165-5749-45cd-885e-bc4fab55f3ef service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:27:28.210777 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 707/1396] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:27:28.578303 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956a32b-b7ab-440e-b0d7-ed66b3fb5685 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:28.578917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956a32b-b7ab-440e-b0d7-ed66b3fb5685 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:28.579034 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956a32b-b7ab-440e-b0d7-ed66b3fb5685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:28.652410 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-0956a32b-b7ab-440e-b0d7-ed66b3fb5685 None None] MFA Rules not processed for user `86300433c724483994f59709a3f015db`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:28.653604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-0956a32b-b7ab-440e-b0d7-ed66b3fb5685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:28.696050 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:28.741539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 690/1397] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 5 14:27:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:28.783598 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690838a-5c95-4113-a96a-31bb3711c224 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:28.783973 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690838a-5c95-4113-a96a-31bb3711c224 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:28.784250 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690838a-5c95-4113-a96a-31bb3711c224 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:28.785769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 708/1398] 10.222.0.15 () {54 vars in 943 bytes} [Mon Sep 5 14:27:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:28.792501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f81e1a-29b7-443c-8e86-9158edac01b9 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:28.792789 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f81e1a-29b7-443c-8e86-9158edac01b9 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:28.793064 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f81e1a-29b7-443c-8e86-9158edac01b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:28.868092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-e8f81e1a-29b7-443c-8e86-9158edac01b9 None None] MFA Rules not processed for user `0a85809d14cb449d848f22a3f39bf641`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:28.869280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e8f81e1a-29b7-443c-8e86-9158edac01b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:28.975033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 691/1399] 10.222.0.15 () {58 vars in 1040 bytes} [Mon Sep 5 14:27:28 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:27:28.984760 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690838a-5c95-4113-a96a-31bb3711c224 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:28.986257 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8690838a-5c95-4113-a96a-31bb3711c224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:29.069428 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:29.128631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690838a-5c95-4113-a96a-31bb3711c224 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:29.230028 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44612480-cdf6-4476-b261-b63f670de1e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a85809d14cb449d848f22a3f39bf641', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:29.230551 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44612480-cdf6-4476-b261-b63f670de1e0 service swift] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:29.230844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44612480-cdf6-4476-b261-b63f670de1e0 service swift] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:29.231115 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44612480-cdf6-4476-b261-b63f670de1e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:29.232721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-44612480-cdf6-4476-b261-b63f670de1e0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:29.294592 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:29.314152 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44612480-cdf6-4476-b261-b63f670de1e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:27:29.315830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44612480-cdf6-4476-b261-b63f670de1e0 service swift] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:27:29.324388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 709/1400] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 5 14:27:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 14:27:30.474450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559f4c8-953e-4241-9601-3dc912b98d54 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:30.474450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559f4c8-953e-4241-9601-3dc912b98d54 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:30.474450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559f4c8-953e-4241-9601-3dc912b98d54 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:30.474450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 692/1401] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:30.475290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae31f0a-c8f5-4592-b587-a95ff7916546 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:30.475290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae31f0a-c8f5-4592-b587-a95ff7916546 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:30.475290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae31f0a-c8f5-4592-b587-a95ff7916546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:30.544135 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-cae31f0a-c8f5-4592-b587-a95ff7916546 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:30.545378 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-cae31f0a-c8f5-4592-b587-a95ff7916546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:30.661159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 710/1402] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:30.947096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559f4c8-953e-4241-9601-3dc912b98d54 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:30.949065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-5559f4c8-953e-4241-9601-3dc912b98d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:30.966059 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:31.118681 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559f4c8-953e-4241-9601-3dc912b98d54 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:31.213290 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:31.244414 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d4a64-704b-45df-939a-72184950fd6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:31.245035 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d4a64-704b-45df-939a-72184950fd6b service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:31.245313 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d4a64-704b-45df-939a-72184950fd6b service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:31.245586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d4a64-704b-45df-939a-72184950fd6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:31.247255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e25d4a64-704b-45df-939a-72184950fd6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:31.353250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d4a64-704b-45df-939a-72184950fd6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:27:31.355215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d4a64-704b-45df-939a-72184950fd6b service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:27:31.375065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 693/1403] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:27:33.485959 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a6902-9b05-4708-88b4-0e8915032c34 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:33.486567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a6902-9b05-4708-88b4-0e8915032c34 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:33.486842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a6902-9b05-4708-88b4-0e8915032c34 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:33.488287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 711/1404] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:33.493563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f0ed7-4dbf-449a-b4a7-2685bfa9ff11 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:33.494000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f0ed7-4dbf-449a-b4a7-2685bfa9ff11 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:33.494119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f0ed7-4dbf-449a-b4a7-2685bfa9ff11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:33.579124 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-ac2f0ed7-4dbf-449a-b4a7-2685bfa9ff11 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:33.581194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ac2f0ed7-4dbf-449a-b4a7-2685bfa9ff11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:33.640505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:33.696375 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 694/1405] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:33.990088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a6902-9b05-4708-88b4-0e8915032c34 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:33.991789 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-594a6902-9b05-4708-88b4-0e8915032c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:34.060261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:34.161193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a6902-9b05-4708-88b4-0e8915032c34 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:34.286092 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b5bcb-b890-4f8e-9db9-fb4f997bf4b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:34.286701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b5bcb-b890-4f8e-9db9-fb4f997bf4b7 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:34.287001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b5bcb-b890-4f8e-9db9-fb4f997bf4b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:34.287267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b5bcb-b890-4f8e-9db9-fb4f997bf4b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:34.288891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ad9b5bcb-b890-4f8e-9db9-fb4f997bf4b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:34.307639 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:34.389137 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b5bcb-b890-4f8e-9db9-fb4f997bf4b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:27:34.390707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b5bcb-b890-4f8e-9db9-fb4f997bf4b7 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:27:34.412392 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 712/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:27:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:27:36.644832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75041619-cba0-47bd-8ce9-41531ee0f97b None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:36.645363 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75041619-cba0-47bd-8ce9-41531ee0f97b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:36.645363 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75041619-cba0-47bd-8ce9-41531ee0f97b None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:36.646839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 695/1407] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:36.652106 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c8f6a-efc2-4eec-a2d6-fd9d94fb4add None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:36.652586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c8f6a-efc2-4eec-a2d6-fd9d94fb4add None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:36.652677 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c8f6a-efc2-4eec-a2d6-fd9d94fb4add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:36.731055 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-c04c8f6a-efc2-4eec-a2d6-fd9d94fb4add None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:36.732278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c04c8f6a-efc2-4eec-a2d6-fd9d94fb4add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:36.832162 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:36.840443 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 713/1408] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:37.131582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75041619-cba0-47bd-8ce9-41531ee0f97b None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:37.133131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-75041619-cba0-47bd-8ce9-41531ee0f97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:37.297173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75041619-cba0-47bd-8ce9-41531ee0f97b None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:37.339991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:37.430807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ef806-5b72-4128-b811-325a5ed1e4ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:37.431337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ef806-5b72-4128-b811-325a5ed1e4ef service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:37.431620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ef806-5b72-4128-b811-325a5ed1e4ef service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:37.431978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ef806-5b72-4128-b811-325a5ed1e4ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:37.433587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9c1ef806-5b72-4128-b811-325a5ed1e4ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:37.530175 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ef806-5b72-4128-b811-325a5ed1e4ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:27:37.532130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ef806-5b72-4128-b811-325a5ed1e4ef service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:27:37.552190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 696/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:27:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:27:46.872268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:47.549616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=83 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:53.219163 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f330c0b-58d8-4233-a8f7-c6d865bc89c5 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:53.219823 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f330c0b-58d8-4233-a8f7-c6d865bc89c5 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:53.219823 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f330c0b-58d8-4233-a8f7-c6d865bc89c5 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:53.221063 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 714/1410] 10.222.0.15 () {54 vars in 918 bytes} [Mon Sep 5 14:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:53.226240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c05ff-c745-4879-a08f-0632c982cf6f None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:53.226643 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c05ff-c745-4879-a08f-0632c982cf6f None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:53.226807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c05ff-c745-4879-a08f-0632c982cf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:53.300639 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-a41c05ff-c745-4879-a08f-0632c982cf6f None None] MFA Rules not processed for user `267825dfb0324be0910203cd5abab377`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:53.301830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a41c05ff-c745-4879-a08f-0632c982cf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:53.405903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 697/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Sep 5 14:27:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:53.426184 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e887a-b55f-46e2-8c22-4c81eb475f4a None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:53.426496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e887a-b55f-46e2-8c22-4c81eb475f4a None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:53.426771 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e887a-b55f-46e2-8c22-4c81eb475f4a None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:53.428241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 715/1412] 10.222.0.15 () {54 vars in 918 bytes} [Mon Sep 5 14:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:53.433019 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eba5c5-7de1-4be2-9a66-407fe17ae679 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:53.433305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eba5c5-7de1-4be2-9a66-407fe17ae679 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:53.433577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eba5c5-7de1-4be2-9a66-407fe17ae679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:53.515622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:53.518344 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-15eba5c5-7de1-4be2-9a66-407fe17ae679 None None] MFA Rules not processed for user `709b9ae82dcd47408fc9a6d54a54428a`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:53.519579 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-15eba5c5-7de1-4be2-9a66-407fe17ae679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:53.632692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 698/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Sep 5 14:27:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:56.115711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6f8d0-ac66-41e5-8582-f52206bea291 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:56.116174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6f8d0-ac66-41e5-8582-f52206bea291 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:56.116294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6f8d0-ac66-41e5-8582-f52206bea291 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:56.117648 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 716/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 14:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:56.122806 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9f642-8b7c-42d7-b9ea-26729922eb9c None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:56.123204 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9f642-8b7c-42d7-b9ea-26729922eb9c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:56.123365 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9f642-8b7c-42d7-b9ea-26729922eb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:56.189440 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-e6a9f642-8b7c-42d7-b9ea-26729922eb9c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:56.190556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e6a9f642-8b7c-42d7-b9ea-26729922eb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:56.244768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:56.286383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 699/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 14:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:56.577869 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6f8d0-ac66-41e5-8582-f52206bea291 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:56.579946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-02d6f8d0-ac66-41e5-8582-f52206bea291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:56.723094 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6f8d0-ac66-41e5-8582-f52206bea291 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:56.793639 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:56.834956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f94cfa-dd4c-4a07-917c-b3cc02781180 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:56.835536 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f94cfa-dd4c-4a07-917c-b3cc02781180 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:56.836007 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f94cfa-dd4c-4a07-917c-b3cc02781180 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:56.836329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f94cfa-dd4c-4a07-917c-b3cc02781180 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:56.838505 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-05f94cfa-dd4c-4a07-917c-b3cc02781180 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:56.925696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f94cfa-dd4c-4a07-917c-b3cc02781180 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:27:56.927314 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f94cfa-dd4c-4a07-917c-b3cc02781180 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:27:56.944837 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 717/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:27:57.823266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3527-915f-415c-92db-6b7526b84ab7 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:57.823812 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3527-915f-415c-92db-6b7526b84ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:57.823812 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3527-915f-415c-92db-6b7526b84ab7 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:57.825172 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 700/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 14:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:57.830460 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a55f37-19c1-4c4e-9493-d11f94251265 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:57.831156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a55f37-19c1-4c4e-9493-d11f94251265 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:57.831298 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a55f37-19c1-4c4e-9493-d11f94251265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:57.905225 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-67a55f37-19c1-4c4e-9493-d11f94251265 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:57.906522 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-67a55f37-19c1-4c4e-9493-d11f94251265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:57.933411 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:58.003554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 718/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 14:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:58.152856 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c3527-915f-415c-92db-6b7526b84ab7 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:58.154420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c20c3527-915f-415c-92db-6b7526b84ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:58.299922 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c3527-915f-415c-92db-6b7526b84ab7 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:58.339139 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:58.410571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695eead-f18a-48a9-b7a6-b83787a7466f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:58.411134 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695eead-f18a-48a9-b7a6-b83787a7466f service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:58.411414 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695eead-f18a-48a9-b7a6-b83787a7466f service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:58.411683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695eead-f18a-48a9-b7a6-b83787a7466f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:58.413280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-0695eead-f18a-48a9-b7a6-b83787a7466f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:58.500466 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0695eead-f18a-48a9-b7a6-b83787a7466f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:27:58.502064 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0695eead-f18a-48a9-b7a6-b83787a7466f service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:27:58.519248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 701/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:27:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:27:59.395284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b272b9-594c-40a9-9939-d5478a561ff0 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:59.395805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b272b9-594c-40a9-9939-d5478a561ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:59.395805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b272b9-594c-40a9-9939-d5478a561ff0 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:59.397151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 719/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 14:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:27:59.402085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdda0f-d6ac-4ee4-8dbe-e7687aff0ef9 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:59.402494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdda0f-d6ac-4ee4-8dbe-e7687aff0ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:59.402623 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdda0f-d6ac-4ee4-8dbe-e7687aff0ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:59.455331 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:59.477117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-b2fdda0f-d6ac-4ee4-8dbe-e7687aff0ef9 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:27:59.478300 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b2fdda0f-d6ac-4ee4-8dbe-e7687aff0ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:59.570805 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 702/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 14:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:27:59.721465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b272b9-594c-40a9-9939-d5478a561ff0 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:27:59.723112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-21b272b9-594c-40a9-9939-d5478a561ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:27:59.865274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b272b9-594c-40a9-9939-d5478a561ff0 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:27:59.877559 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:27:59.977830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99c695-e921-4a63-82b9-e4d588ce9d99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:27:59.978343 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99c695-e921-4a63-82b9-e4d588ce9d99 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:27:59.978618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99c695-e921-4a63-82b9-e4d588ce9d99 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:27:59.978914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99c695-e921-4a63-82b9-e4d588ce9d99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:27:59.980492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ce99c695-e921-4a63-82b9-e4d588ce9d99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:00.068996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce99c695-e921-4a63-82b9-e4d588ce9d99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:00.070596 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce99c695-e921-4a63-82b9-e4d588ce9d99 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:00.089198 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 720/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:28:00.959008 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795757c8-159d-4d95-8b11-511663ea0225 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:00.959585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795757c8-159d-4d95-8b11-511663ea0225 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:00.959585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795757c8-159d-4d95-8b11-511663ea0225 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:00.960896 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 703/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 14:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:00.965626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe8e32-b579-4b82-ac60-3a4d0cc5e7aa None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:00.966082 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe8e32-b579-4b82-ac60-3a4d0cc5e7aa None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:00.966182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe8e32-b579-4b82-ac60-3a4d0cc5e7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:00.986052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:01.041133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-2abe8e32-b579-4b82-ac60-3a4d0cc5e7aa None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:01.042326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-2abe8e32-b579-4b82-ac60-3a4d0cc5e7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:01.136712 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 721/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 14:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:28:01.285484 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795757c8-159d-4d95-8b11-511663ea0225 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:01.287039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-795757c8-159d-4d95-8b11-511663ea0225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:01.398008 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:01.432626 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795757c8-159d-4d95-8b11-511663ea0225 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:28:01.547126 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c827c-7c1d-4956-86d4-95664116f1c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:28:01.547126 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c827c-7c1d-4956-86d4-95664116f1c2 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:01.547126 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c827c-7c1d-4956-86d4-95664116f1c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:01.547126 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c827c-7c1d-4956-86d4-95664116f1c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:01.548680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-6e6c827c-7c1d-4956-86d4-95664116f1c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:01.639554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c827c-7c1d-4956-86d4-95664116f1c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:01.641235 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c827c-7c1d-4956-86d4-95664116f1c2 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:01.647623 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:01.660450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 704/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:28:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:28:02.511574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d07ca-7550-47c2-a4b6-043c82ff65cb None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:02.512180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d07ca-7550-47c2-a4b6-043c82ff65cb None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:02.512180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d07ca-7550-47c2-a4b6-043c82ff65cb None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:02.513464 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 722/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 14:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:02.519028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eedca5-73b4-4278-bfaa-f3dfcd504e0c None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:02.519482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eedca5-73b4-4278-bfaa-f3dfcd504e0c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:02.519582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eedca5-73b4-4278-bfaa-f3dfcd504e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:02.595524 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-14eedca5-73b4-4278-bfaa-f3dfcd504e0c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:02.596803 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-14eedca5-73b4-4278-bfaa-f3dfcd504e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:02.691626 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 705/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 14:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:28:02.843031 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d07ca-7550-47c2-a4b6-043c82ff65cb None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:02.845113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-df4d07ca-7550-47c2-a4b6-043c82ff65cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:02.928020 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:02.988669 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d07ca-7550-47c2-a4b6-043c82ff65cb None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:28:03.101059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9aeeb-f7d6-4943-91f4-40dab54e9692 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:28:03.101674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9aeeb-f7d6-4943-91f4-40dab54e9692 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:03.101954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9aeeb-f7d6-4943-91f4-40dab54e9692 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:03.102226 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9aeeb-f7d6-4943-91f4-40dab54e9692 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:03.103861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-12e9aeeb-f7d6-4943-91f4-40dab54e9692 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:03.151999 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:03.193204 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9aeeb-f7d6-4943-91f4-40dab54e9692 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:03.194828 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9aeeb-f7d6-4943-91f4-40dab54e9692 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:03.213266 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 723/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:28:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:28:04.077114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50483860-fdfd-47df-a48a-1362ef764a4d None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:04.077655 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50483860-fdfd-47df-a48a-1362ef764a4d None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:04.077810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50483860-fdfd-47df-a48a-1362ef764a4d None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:04.079366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 706/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 14:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:04.084595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f99eb-08b5-40fa-bd65-c25d5b874593 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:04.085029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f99eb-08b5-40fa-bd65-c25d5b874593 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:04.085134 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f99eb-08b5-40fa-bd65-c25d5b874593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:04.160516 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-1d3f99eb-08b5-40fa-bd65-c25d5b874593 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:04.161660 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-1d3f99eb-08b5-40fa-bd65-c25d5b874593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:04.257702 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 724/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 14:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:28:04.406944 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50483860-fdfd-47df-a48a-1362ef764a4d None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:04.408455 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-50483860-fdfd-47df-a48a-1362ef764a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:04.456846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:04.551538 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50483860-fdfd-47df-a48a-1362ef764a4d None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:28:04.663394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f261da-c7ef-435f-89e1-a454afd32eaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:28:04.664003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f261da-c7ef-435f-89e1-a454afd32eaf service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:04.664285 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f261da-c7ef-435f-89e1-a454afd32eaf service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:04.664562 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f261da-c7ef-435f-89e1-a454afd32eaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:04.666155 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-58f261da-c7ef-435f-89e1-a454afd32eaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:04.673115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:04.754037 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f261da-c7ef-435f-89e1-a454afd32eaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:04.755651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f261da-c7ef-435f-89e1-a454afd32eaf service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:04.773509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 707/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:28:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:28:05.628499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62460c6a-2982-4b4f-923d-040ee1e4f4f3 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:05.629066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62460c6a-2982-4b4f-923d-040ee1e4f4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:05.629066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62460c6a-2982-4b4f-923d-040ee1e4f4f3 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:05.630393 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 725/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 14:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:05.635386 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f693641-2bc7-4ae4-b455-41ee64f07531 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:05.635814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f693641-2bc7-4ae4-b455-41ee64f07531 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:05.635930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f693641-2bc7-4ae4-b455-41ee64f07531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:05.710056 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-4f693641-2bc7-4ae4-b455-41ee64f07531 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:05.711302 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-4f693641-2bc7-4ae4-b455-41ee64f07531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:05.776946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:05.805644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 708/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 14:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:28:05.953070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62460c6a-2982-4b4f-923d-040ee1e4f4f3 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:05.954525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-62460c6a-2982-4b4f-923d-040ee1e4f4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:05.972701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:06.099849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62460c6a-2982-4b4f-923d-040ee1e4f4f3 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:28:06.186555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:06.212175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa42159-fcfa-4cc0-9c84-cb522fd86a1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:28:06.212729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa42159-fcfa-4cc0-9c84-cb522fd86a1c service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:06.213008 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa42159-fcfa-4cc0-9c84-cb522fd86a1c service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:06.213279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa42159-fcfa-4cc0-9c84-cb522fd86a1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:06.214914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-3aa42159-fcfa-4cc0-9c84-cb522fd86a1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:06.304656 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa42159-fcfa-4cc0-9c84-cb522fd86a1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:06.306279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa42159-fcfa-4cc0-9c84-cb522fd86a1c service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:06.324229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 726/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:28:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:28:07.303852 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aec72-0330-48f3-9233-5d275b9c83ef None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:07.304386 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aec72-0330-48f3-9233-5d275b9c83ef None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:07.304702 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aec72-0330-48f3-9233-5d275b9c83ef None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:07.306161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 709/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 14:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:07.311263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874b6ac-e5eb-4c52-8455-5de747dc88cb None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:07.311699 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874b6ac-e5eb-4c52-8455-5de747dc88cb None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:07.311797 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874b6ac-e5eb-4c52-8455-5de747dc88cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:07.387154 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-c874b6ac-e5eb-4c52-8455-5de747dc88cb None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:07.388279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c874b6ac-e5eb-4c52-8455-5de747dc88cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:07.428856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:07.481606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 727/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 14:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:28:07.630360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99aec72-0330-48f3-9233-5d275b9c83ef None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:07.632013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a99aec72-0330-48f3-9233-5d275b9c83ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:07.774896 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99aec72-0330-48f3-9233-5d275b9c83ef None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:28:07.826904 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:07.883898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f5696-a5a0-4e0e-b099-a66041b01e72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:28:07.884495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f5696-a5a0-4e0e-b099-a66041b01e72 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:07.884771 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f5696-a5a0-4e0e-b099-a66041b01e72 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:07.885040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f5696-a5a0-4e0e-b099-a66041b01e72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:07.886614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-005f5696-a5a0-4e0e-b099-a66041b01e72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:07.974320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f5696-a5a0-4e0e-b099-a66041b01e72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:07.975926 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f5696-a5a0-4e0e-b099-a66041b01e72 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:07.993089 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 710/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:28:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:28:08.850163 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00253d86-ae71-47c7-8a20-257c1caffe39 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:08.850738 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00253d86-ae71-47c7-8a20-257c1caffe39 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:08.850738 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00253d86-ae71-47c7-8a20-257c1caffe39 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:08.852095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 728/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 14:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:08.857028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ce2f5-1dc6-4340-9493-4155b299ff5c None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:08.857450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ce2f5-1dc6-4340-9493-4155b299ff5c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:08.857578 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ce2f5-1dc6-4340-9493-4155b299ff5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:08.931538 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-be4ce2f5-1dc6-4340-9493-4155b299ff5c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:08.932740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-be4ce2f5-1dc6-4340-9493-4155b299ff5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:08.934719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:09.027636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 711/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 14:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:28:09.174891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00253d86-ae71-47c7-8a20-257c1caffe39 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:09.176335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-00253d86-ae71-47c7-8a20-257c1caffe39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:09.325507 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00253d86-ae71-47c7-8a20-257c1caffe39 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:28:09.353183 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:09.439661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305267ee-1e3c-483e-84e2-333f06edaed0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:28:09.440196 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305267ee-1e3c-483e-84e2-333f06edaed0 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:09.440472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305267ee-1e3c-483e-84e2-333f06edaed0 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:09.440741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305267ee-1e3c-483e-84e2-333f06edaed0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:09.442338 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-305267ee-1e3c-483e-84e2-333f06edaed0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:09.529079 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305267ee-1e3c-483e-84e2-333f06edaed0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:09.530652 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305267ee-1e3c-483e-84e2-333f06edaed0 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:09.548371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 729/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:28:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:28:11.352972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373bce5-581c-4d30-8b5a-76ddcdcef26a None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:11.353482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373bce5-581c-4d30-8b5a-76ddcdcef26a None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:11.353482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373bce5-581c-4d30-8b5a-76ddcdcef26a None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:11.354823 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 712/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 14:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:11.360720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce357e-c7e3-4f11-ba17-bc8b833edaae None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:11.362256 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce357e-c7e3-4f11-ba17-bc8b833edaae None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:11.362256 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce357e-c7e3-4f11-ba17-bc8b833edaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:11.402330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:11.427624 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-a5ce357e-c7e3-4f11-ba17-bc8b833edaae None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:11.428819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a5ce357e-c7e3-4f11-ba17-bc8b833edaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:11.524364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 730/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 14:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:28:11.812531 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373bce5-581c-4d30-8b5a-76ddcdcef26a None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:11.814641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f373bce5-581c-4d30-8b5a-76ddcdcef26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:11.915637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373bce5-581c-4d30-8b5a-76ddcdcef26a None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:28:11.947198 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:11.987511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1ac16-fb7c-4a08-b32c-af96638fa73b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:28:11.988121 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1ac16-fb7c-4a08-b32c-af96638fa73b service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:11.988399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1ac16-fb7c-4a08-b32c-af96638fa73b service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:11.988669 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1ac16-fb7c-4a08-b32c-af96638fa73b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:11.990252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-dcd1ac16-fb7c-4a08-b32c-af96638fa73b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:12.098445 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1ac16-fb7c-4a08-b32c-af96638fa73b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:12.099924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1ac16-fb7c-4a08-b32c-af96638fa73b service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:12.118668 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 713/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:28:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:28:14.152867 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b1f02-fbae-4b93-aad9-e53d507f7faa None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:14.153396 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b1f02-fbae-4b93-aad9-e53d507f7faa None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:14.153396 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b1f02-fbae-4b93-aad9-e53d507f7faa None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:14.154708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 731/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 14:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:14.159831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18c5fd-a1d7-4188-a401-b05565a48453 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:14.160237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18c5fd-a1d7-4188-a401-b05565a48453 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:14.160407 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18c5fd-a1d7-4188-a401-b05565a48453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:14.226209 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-cb18c5fd-a1d7-4188-a401-b05565a48453 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:14.227449 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-cb18c5fd-a1d7-4188-a401-b05565a48453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:14.239973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:14.324189 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 714/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 14:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:28:14.609498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b1f02-fbae-4b93-aad9-e53d507f7faa None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:14.611597 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-672b1f02-fbae-4b93-aad9-e53d507f7faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:14.715714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b1f02-fbae-4b93-aad9-e53d507f7faa None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:28:14.728184 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:14.788273 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f237d8-c01b-40f1-b0f4-987e7d680e6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:28:14.788882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f237d8-c01b-40f1-b0f4-987e7d680e6c service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:14.789169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f237d8-c01b-40f1-b0f4-987e7d680e6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:14.789442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f237d8-c01b-40f1-b0f4-987e7d680e6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:14.791071 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-87f237d8-c01b-40f1-b0f4-987e7d680e6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:14.901705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f237d8-c01b-40f1-b0f4-987e7d680e6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:14.903083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f237d8-c01b-40f1-b0f4-987e7d680e6c service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:14.920903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 732/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:28:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:28:17.217365 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c7b33-c4b7-4590-95c6-b92ed7f1d2dc None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:17.217937 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c7b33-c4b7-4590-95c6-b92ed7f1d2dc None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:17.217937 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c7b33-c4b7-4590-95c6-b92ed7f1d2dc None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:17.219371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 715/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 14:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:17.224494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f82b4-0b7c-4fb0-8abc-22d76c94edfd None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:17.224917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f82b4-0b7c-4fb0-8abc-22d76c94edfd None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:17.225027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f82b4-0b7c-4fb0-8abc-22d76c94edfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:17.271993 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:17.289555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-d63f82b4-0b7c-4fb0-8abc-22d76c94edfd None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:17.290757 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d63f82b4-0b7c-4fb0-8abc-22d76c94edfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:17.387005 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 733/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 14:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:28:17.671784 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c7b33-c4b7-4590-95c6-b92ed7f1d2dc None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:17.673929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8c8c7b33-c4b7-4590-95c6-b92ed7f1d2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:17.774664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c7b33-c4b7-4590-95c6-b92ed7f1d2dc None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:28:17.812720 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:17.845749 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8042e7-21a1-47a9-aa8c-fd39c1979a5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:28:17.846352 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8042e7-21a1-47a9-aa8c-fd39c1979a5c service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:17.846625 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8042e7-21a1-47a9-aa8c-fd39c1979a5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:17.846923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8042e7-21a1-47a9-aa8c-fd39c1979a5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:17.848503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-2a8042e7-21a1-47a9-aa8c-fd39c1979a5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:17.954478 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8042e7-21a1-47a9-aa8c-fd39c1979a5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:17.956125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8042e7-21a1-47a9-aa8c-fd39c1979a5c service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:17.975616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 716/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:28:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:28:19.997650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda44b7-1f78-48cb-8f7d-73bb5d75ad7b None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:19.998200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda44b7-1f78-48cb-8f7d-73bb5d75ad7b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:19.998200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda44b7-1f78-48cb-8f7d-73bb5d75ad7b None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:19.999585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 734/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 14:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:20.006008 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed5801-4548-4f7c-bc51-8d177221afe9 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:20.006008 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed5801-4548-4f7c-bc51-8d177221afe9 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:20.006008 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed5801-4548-4f7c-bc51-8d177221afe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:20.070960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-2bed5801-4548-4f7c-bc51-8d177221afe9 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:20.072164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-2bed5801-4548-4f7c-bc51-8d177221afe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:20.093509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:20.169056 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 717/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 14:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:28:20.464245 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda44b7-1f78-48cb-8f7d-73bb5d75ad7b None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:20.465788 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0dda44b7-1f78-48cb-8f7d-73bb5d75ad7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:20.567888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda44b7-1f78-48cb-8f7d-73bb5d75ad7b None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:28:20.587881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:20.640750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e013ec2-4bb6-4a84-9027-b2f184263321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:28:20.641904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e013ec2-4bb6-4a84-9027-b2f184263321 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:20.642181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e013ec2-4bb6-4a84-9027-b2f184263321 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:20.642450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e013ec2-4bb6-4a84-9027-b2f184263321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:20.644097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-4e013ec2-4bb6-4a84-9027-b2f184263321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:20.756432 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e013ec2-4bb6-4a84-9027-b2f184263321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:20.757884 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e013ec2-4bb6-4a84-9027-b2f184263321 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:20.776160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 735/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:28:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:28:22.819661 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f201b-35a2-416d-aaf7-5eabb67380b5 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:22.820206 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f201b-35a2-416d-aaf7-5eabb67380b5 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:22.820206 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f201b-35a2-416d-aaf7-5eabb67380b5 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:22.821569 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 718/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 14:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:22.826768 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828451d6-f35e-49e0-a9f3-08983be5fb2b None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:22.827229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828451d6-f35e-49e0-a9f3-08983be5fb2b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:22.827336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828451d6-f35e-49e0-a9f3-08983be5fb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:22.892361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:22.895038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-828451d6-f35e-49e0-a9f3-08983be5fb2b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:22.896251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-828451d6-f35e-49e0-a9f3-08983be5fb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:22.990494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 736/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 14:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:28:23.271445 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f201b-35a2-416d-aaf7-5eabb67380b5 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:23.272978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-248f201b-35a2-416d-aaf7-5eabb67380b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:23.376240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f201b-35a2-416d-aaf7-5eabb67380b5 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:28:23.424637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:23.448336 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881942df-c5b3-4ffe-bd94-7133e8190233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:28:23.448944 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881942df-c5b3-4ffe-bd94-7133e8190233 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:23.449225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881942df-c5b3-4ffe-bd94-7133e8190233 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:23.449506 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881942df-c5b3-4ffe-bd94-7133e8190233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:23.451172 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-881942df-c5b3-4ffe-bd94-7133e8190233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:23.557431 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881942df-c5b3-4ffe-bd94-7133e8190233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:23.558884 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881942df-c5b3-4ffe-bd94-7133e8190233 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:23.576716 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 719/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:28:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:28:25.530971 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6e17b-a970-4961-b02f-d7d0cf491554 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:25.532054 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6e17b-a970-4961-b02f-d7d0cf491554 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:25.532170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6e17b-a970-4961-b02f-d7d0cf491554 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:25.533493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 737/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 14:28:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:25.538768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf7763-5bf4-4cad-8a07-e5eeae7e1590 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:25.539213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf7763-5bf4-4cad-8a07-e5eeae7e1590 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:25.539364 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf7763-5bf4-4cad-8a07-e5eeae7e1590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:25.605761 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-c0cf7763-5bf4-4cad-8a07-e5eeae7e1590 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:25.607002 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c0cf7763-5bf4-4cad-8a07-e5eeae7e1590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:25.635061 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:25.704990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 720/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 14:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:28:25.988204 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6e17b-a970-4961-b02f-d7d0cf491554 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:25.989741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-4af6e17b-a970-4961-b02f-d7d0cf491554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:26.090925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6e17b-a970-4961-b02f-d7d0cf491554 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:28:26.119609 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:26.163698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3b3cf-4caf-4132-81f1-e9cfabe49800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:28:26.164457 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3b3cf-4caf-4132-81f1-e9cfabe49800 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:26.164671 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3b3cf-4caf-4132-81f1-e9cfabe49800 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:26.164938 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3b3cf-4caf-4132-81f1-e9cfabe49800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:26.166520 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-04a3b3cf-4caf-4132-81f1-e9cfabe49800 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:26.271967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3b3cf-4caf-4132-81f1-e9cfabe49800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:26.273403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3b3cf-4caf-4132-81f1-e9cfabe49800 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:26.291571 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 738/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:28:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:28:28.247051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed647f-5de6-4f9b-b9e7-31e2e5db5a40 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:28.247600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed647f-5de6-4f9b-b9e7-31e2e5db5a40 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:28.247600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed647f-5de6-4f9b-b9e7-31e2e5db5a40 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:28.248984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 721/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 14:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:28.254746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4461342-eeff-439d-9376-2d90b34d628c None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:28.255204 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4461342-eeff-439d-9376-2d90b34d628c None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:28.255334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4461342-eeff-439d-9376-2d90b34d628c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:28.321385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-a4461342-eeff-439d-9376-2d90b34d628c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:28.322527 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a4461342-eeff-439d-9376-2d90b34d628c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:28.331688 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:28.419827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 739/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 14:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:28:28.700602 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed647f-5de6-4f9b-b9e7-31e2e5db5a40 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:28.702149 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ffed647f-5de6-4f9b-b9e7-31e2e5db5a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:28.804712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed647f-5de6-4f9b-b9e7-31e2e5db5a40 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:28:28.858264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:28.876079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320bfd16-956d-4a29-9836-d0863ee48f6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:28:28.877231 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bfd16-956d-4a29-9836-d0863ee48f6f service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:28.877509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bfd16-956d-4a29-9836-d0863ee48f6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:28.877774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bfd16-956d-4a29-9836-d0863ee48f6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:28.879374 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-320bfd16-956d-4a29-9836-d0863ee48f6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:28.984333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320bfd16-956d-4a29-9836-d0863ee48f6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:28.985818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320bfd16-956d-4a29-9836-d0863ee48f6f service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:29.004872 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 722/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:28:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:28:30.944329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27653eb8-dc81-4ec7-a249-1b9524afa7c1 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:30.944899 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27653eb8-dc81-4ec7-a249-1b9524afa7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:30.944899 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27653eb8-dc81-4ec7-a249-1b9524afa7c1 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:30.946222 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 740/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 14:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:30.951441 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0750a2-1126-410a-a374-15f48a18543b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:30.951855 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0750a2-1126-410a-a374-15f48a18543b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:30.951995 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0750a2-1126-410a-a374-15f48a18543b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:31.018184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-ac0750a2-1126-410a-a374-15f48a18543b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:31.019414 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ac0750a2-1126-410a-a374-15f48a18543b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:31.056971 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:31.114665 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 723/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 14:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:28:31.400312 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27653eb8-dc81-4ec7-a249-1b9524afa7c1 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:31.401915 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-27653eb8-dc81-4ec7-a249-1b9524afa7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:31.505840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27653eb8-dc81-4ec7-a249-1b9524afa7c1 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:28:31.542757 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:31.578366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf44ce-3779-4f02-98ef-13645381b659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:28:31.579002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf44ce-3779-4f02-98ef-13645381b659 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:31.579289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf44ce-3779-4f02-98ef-13645381b659 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:31.579579 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf44ce-3779-4f02-98ef-13645381b659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:31.581430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-aadf44ce-3779-4f02-98ef-13645381b659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:31.688801 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf44ce-3779-4f02-98ef-13645381b659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:31.690809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf44ce-3779-4f02-98ef-13645381b659 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:31.709632 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 741/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:28:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:28:33.632632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7f79a-13a8-4355-9739-2e40f1776eb9 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:33.633110 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7f79a-13a8-4355-9739-2e40f1776eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:33.633220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7f79a-13a8-4355-9739-2e40f1776eb9 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:33.634510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 724/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 14:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:33.639898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc89766-8e13-492b-aa59-2013bdb9f9b8 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:33.640319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc89766-8e13-492b-aa59-2013bdb9f9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:33.640451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc89766-8e13-492b-aa59-2013bdb9f9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:33.706573 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-5bc89766-8e13-492b-aa59-2013bdb9f9b8 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:33.707825 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5bc89766-8e13-492b-aa59-2013bdb9f9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:33.726486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:33.804812 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 742/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 14:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:28:34.089119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7f79a-13a8-4355-9739-2e40f1776eb9 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:34.090749 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-62d7f79a-13a8-4355-9739-2e40f1776eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:34.192768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7f79a-13a8-4355-9739-2e40f1776eb9 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:28:34.258285 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:34.264606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812fa056-fb00-4a55-af6e-874ab7698d1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:28:34.265214 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fa056-fb00-4a55-af6e-874ab7698d1c service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:34.265494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fa056-fb00-4a55-af6e-874ab7698d1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:34.265763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fa056-fb00-4a55-af6e-874ab7698d1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:34.268014 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-812fa056-fb00-4a55-af6e-874ab7698d1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:34.374099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812fa056-fb00-4a55-af6e-874ab7698d1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:34.375575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812fa056-fb00-4a55-af6e-874ab7698d1c service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:34.394158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 725/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:28:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:28:36.306488 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1663c0f-23be-4937-afde-569e5bceb1f8 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:36.307064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1663c0f-23be-4937-afde-569e5bceb1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:36.307064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1663c0f-23be-4937-afde-569e5bceb1f8 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:36.308426 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 743/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 14:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:36.313819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ee613-dea4-45fa-8b0e-ccbaf7175ff0 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:36.314341 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ee613-dea4-45fa-8b0e-ccbaf7175ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:36.314448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ee613-dea4-45fa-8b0e-ccbaf7175ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:36.382536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-a23ee613-dea4-45fa-8b0e-ccbaf7175ff0 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:36.383825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a23ee613-dea4-45fa-8b0e-ccbaf7175ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:36.433352 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:36.483835 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 726/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 14:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:28:36.766546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1663c0f-23be-4937-afde-569e5bceb1f8 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:36.768694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e1663c0f-23be-4937-afde-569e5bceb1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:36.988312 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1663c0f-23be-4937-afde-569e5bceb1f8 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:28:37.034240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:37.058683 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50ceab-793b-4a4f-b7d8-24d09068c404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:28:37.059300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50ceab-793b-4a4f-b7d8-24d09068c404 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:37.059577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50ceab-793b-4a4f-b7d8-24d09068c404 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:37.059843 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50ceab-793b-4a4f-b7d8-24d09068c404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:37.061414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8d50ceab-793b-4a4f-b7d8-24d09068c404 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:37.167650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50ceab-793b-4a4f-b7d8-24d09068c404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:37.169077 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50ceab-793b-4a4f-b7d8-24d09068c404 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:37.187265 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 744/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:28:46.481044 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=22 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:47.184986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=66 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:56.562905 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e6de9-b438-422a-83f4-b0ea2cb6ae06 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:56.563470 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e6de9-b438-422a-83f4-b0ea2cb6ae06 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:56.563570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e6de9-b438-422a-83f4-b0ea2cb6ae06 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:56.564891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 727/1471] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:56.569946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b6716-ed6d-46e0-af69-de84d3cf0a5b None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:56.570385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b6716-ed6d-46e0-af69-de84d3cf0a5b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:56.570490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b6716-ed6d-46e0-af69-de84d3cf0a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:56.635510 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-1c2b6716-ed6d-46e0-af69-de84d3cf0a5b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:56.636697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-1c2b6716-ed6d-46e0-af69-de84d3cf0a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:56.731835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 745/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:28:57.031805 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1694f8-e061-48f3-94b3-9cf2c2a2abdb None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:57.032091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1694f8-e061-48f3-94b3-9cf2c2a2abdb None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:57.033077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1694f8-e061-48f3-94b3-9cf2c2a2abdb None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:57.034399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 728/1473] 10.222.0.15 () {54 vars in 961 bytes} [Mon Sep 5 14:28:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:57.039235 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7ad6d-3d85-4fab-a4d9-3c43d84766d2 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:57.039524 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7ad6d-3d85-4fab-a4d9-3c43d84766d2 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:57.039802 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7ad6d-3d85-4fab-a4d9-3c43d84766d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:57.115849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-51c7ad6d-3d85-4fab-a4d9-3c43d84766d2 None None] MFA Rules not processed for user `735986478c354ed196c601f12f3ac377`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:57.117047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-51c7ad6d-3d85-4fab-a4d9-3c43d84766d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:57.136792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:57.191319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 746/1474] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Sep 5 14:28:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:28:57.198737 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1694f8-e061-48f3-94b3-9cf2c2a2abdb None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:57.200275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ca1694f8-e061-48f3-94b3-9cf2c2a2abdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:57.301459 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1694f8-e061-48f3-94b3-9cf2c2a2abdb None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:28:57.374741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2aa06-94e2-417e-96e3-f375bc433809 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '735986478c354ed196c601f12f3ac377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:28:57.375370 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2aa06-94e2-417e-96e3-f375bc433809 service glance] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:57.375647 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2aa06-94e2-417e-96e3-f375bc433809 service glance] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:57.376449 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2aa06-94e2-417e-96e3-f375bc433809 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:57.378108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e9e2aa06-94e2-417e-96e3-f375bc433809 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:57.430613 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:28:57.484672 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2aa06-94e2-417e-96e3-f375bc433809 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:28:57.486153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2aa06-94e2-417e-96e3-f375bc433809 service glance] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:28:57.504652 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 729/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:28:59.497085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29214cc-abf4-4602-a65c-cad9ec0850fa None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:59.497572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29214cc-abf4-4602-a65c-cad9ec0850fa None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:59.497686 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29214cc-abf4-4602-a65c-cad9ec0850fa None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:59.499019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 747/1476] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:28:59.504927 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0468871-9ffd-4d87-aa71-3cd80029f8f8 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:28:59.505330 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0468871-9ffd-4d87-aa71-3cd80029f8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:28:59.505484 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0468871-9ffd-4d87-aa71-3cd80029f8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:28:59.570433 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-c0468871-9ffd-4d87-aa71-3cd80029f8f8 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:28:59.571675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c0468871-9ffd-4d87-aa71-3cd80029f8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:28:59.665996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 730/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:28:59.939848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29214cc-abf4-4602-a65c-cad9ec0850fa None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:28:59.941288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c29214cc-abf4-4602-a65c-cad9ec0850fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:29:00.063735 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29214cc-abf4-4602-a65c-cad9ec0850fa None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:29:00.112900 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:29:00.158085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22577ec-ca2f-4694-a8ae-1c066bc03485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:29:00.158698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22577ec-ca2f-4694-a8ae-1c066bc03485 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:29:00.159009 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22577ec-ca2f-4694-a8ae-1c066bc03485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:29:00.159283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22577ec-ca2f-4694-a8ae-1c066bc03485 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:29:00.159922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22577ec-ca2f-4694-a8ae-1c066bc03485 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:29:00.161660 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: WARNING py.warnings [None req-f22577ec-ca2f-4694-a8ae-1c066bc03485 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 14:29:00.161660 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: warnings.warn(msg) Sep 05 14:29:00.161660 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: Sep 05 14:29:00.162815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22577ec-ca2f-4694-a8ae-1c066bc03485 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:29:00.181617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 748/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 5 14:28:59 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:29:02.080675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a8f7f-ed80-49b5-a6cb-da4ac3140f17 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:29:02.081205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a8f7f-ed80-49b5-a6cb-da4ac3140f17 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:29:02.081402 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a8f7f-ed80-49b5-a6cb-da4ac3140f17 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:29:02.082895 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 731/1479] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:29:02.087920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75b26f-a4bf-4ebe-9ec7-79b2df2c1f78 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:29:02.088334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75b26f-a4bf-4ebe-9ec7-79b2df2c1f78 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:29:02.088503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75b26f-a4bf-4ebe-9ec7-79b2df2c1f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:29:02.152669 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-7a75b26f-a4bf-4ebe-9ec7-79b2df2c1f78 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:29:02.153858 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7a75b26f-a4bf-4ebe-9ec7-79b2df2c1f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:29:02.248271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 749/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:29:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:29:02.554621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a8f7f-ed80-49b5-a6cb-da4ac3140f17 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:29:02.556178 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-0d6a8f7f-ed80-49b5-a6cb-da4ac3140f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:29:02.566067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:29:02.700323 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a8f7f-ed80-49b5-a6cb-da4ac3140f17 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:29:02.780657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:29:02.812130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213018f0-ee64-4877-83a6-1d48a2005db6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:29:02.812678 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213018f0-ee64-4877-83a6-1d48a2005db6 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:29:02.813168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213018f0-ee64-4877-83a6-1d48a2005db6 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:29:02.814269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213018f0-ee64-4877-83a6-1d48a2005db6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:29:02.815920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-213018f0-ee64-4877-83a6-1d48a2005db6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:29:02.914200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213018f0-ee64-4877-83a6-1d48a2005db6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:29:02.915822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213018f0-ee64-4877-83a6-1d48a2005db6 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:29:02.933078 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 732/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:29:05.030085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6b17c-1d75-45b9-b143-c9c7bf6ddb10 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:29:05.030690 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6b17c-1d75-45b9-b143-c9c7bf6ddb10 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:29:05.030690 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6b17c-1d75-45b9-b143-c9c7bf6ddb10 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:29:05.031997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 750/1482] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:29:05.037605 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84ee5f-e0ef-4faf-ac88-36fb67dfeb65 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:29:05.038044 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84ee5f-e0ef-4faf-ac88-36fb67dfeb65 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:29:05.038189 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84ee5f-e0ef-4faf-ac88-36fb67dfeb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:29:05.105095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-6a84ee5f-e0ef-4faf-ac88-36fb67dfeb65 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:29:05.106306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-6a84ee5f-e0ef-4faf-ac88-36fb67dfeb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:29:05.152504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:29:05.202606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 733/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:29:05.490486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6b17c-1d75-45b9-b143-c9c7bf6ddb10 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:29:05.492049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-44b6b17c-1d75-45b9-b143-c9c7bf6ddb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:29:05.515099 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:29:05.594415 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6b17c-1d75-45b9-b143-c9c7bf6ddb10 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:29:05.666517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695e812-4b34-471d-aab9-75b08a00ba05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:29:05.667143 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695e812-4b34-471d-aab9-75b08a00ba05 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:29:05.667420 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695e812-4b34-471d-aab9-75b08a00ba05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:29:05.667685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695e812-4b34-471d-aab9-75b08a00ba05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:29:05.669260 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-4695e812-4b34-471d-aab9-75b08a00ba05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:29:05.742106 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:29:05.774890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695e812-4b34-471d-aab9-75b08a00ba05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:29:05.776318 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695e812-4b34-471d-aab9-75b08a00ba05 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:29:05.794457 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 751/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:29:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:29:07.748661 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bd658-fa29-4540-b9e9-c15855499a55 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:29:07.749216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bd658-fa29-4540-b9e9-c15855499a55 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:29:07.749331 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bd658-fa29-4540-b9e9-c15855499a55 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:29:07.750588 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 734/1485] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:29:07.755820 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db174f1-9622-4900-a284-278c2ca6ebd7 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:29:07.756247 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db174f1-9622-4900-a284-278c2ca6ebd7 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:29:07.756373 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db174f1-9622-4900-a284-278c2ca6ebd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:29:07.821210 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-8db174f1-9622-4900-a284-278c2ca6ebd7 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:29:07.822351 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8db174f1-9622-4900-a284-278c2ca6ebd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:29:07.918452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 752/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:29:08.203273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bd658-fa29-4540-b9e9-c15855499a55 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:29:08.205455 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-fd9bd658-fa29-4540-b9e9-c15855499a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:29:08.312307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bd658-fa29-4540-b9e9-c15855499a55 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:29:08.384681 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:29:08.386349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631a0c8-685c-4724-8695-016ecb557e11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:29:08.386985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631a0c8-685c-4724-8695-016ecb557e11 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:29:08.387271 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631a0c8-685c-4724-8695-016ecb557e11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:29:08.387569 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631a0c8-685c-4724-8695-016ecb557e11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:29:08.389191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e631a0c8-685c-4724-8695-016ecb557e11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:29:08.496703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631a0c8-685c-4724-8695-016ecb557e11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:29:08.498150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631a0c8-685c-4724-8695-016ecb557e11 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:29:08.516293 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 735/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:29:10.496978 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a1596-dec4-4309-b170-7be5464e2970 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:29:10.497506 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a1596-dec4-4309-b170-7be5464e2970 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:29:10.497506 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a1596-dec4-4309-b170-7be5464e2970 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:29:10.498886 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 753/1488] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:29:10.504554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b461a-2a64-48f1-9740-7c39e0137165 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:29:10.504963 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b461a-2a64-48f1-9740-7c39e0137165 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:29:10.505108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b461a-2a64-48f1-9740-7c39e0137165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:29:10.570088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-e74b461a-2a64-48f1-9740-7c39e0137165 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:29:10.571335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e74b461a-2a64-48f1-9740-7c39e0137165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:29:10.622505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:29:10.666547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 736/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:29:10.953444 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a1596-dec4-4309-b170-7be5464e2970 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:29:10.955361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7e1a1596-dec4-4309-b170-7be5464e2970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:29:10.991873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:29:11.060361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a1596-dec4-4309-b170-7be5464e2970 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:29:11.136042 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3b399-71b8-4570-95ca-d0af88385e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:29:11.136645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3b399-71b8-4570-95ca-d0af88385e2b service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:29:11.136924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3b399-71b8-4570-95ca-d0af88385e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:29:11.137195 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3b399-71b8-4570-95ca-d0af88385e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:29:11.138781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-08c3b399-71b8-4570-95ca-d0af88385e2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:29:11.216612 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:29:11.245313 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3b399-71b8-4570-95ca-d0af88385e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:29:11.246814 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3b399-71b8-4570-95ca-d0af88385e2b service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:29:11.265284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 754/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:29:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:29:13.517408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06deefa0-e424-4c78-9516-0bc6f42d1d55 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:29:13.517965 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06deefa0-e424-4c78-9516-0bc6f42d1d55 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:29:13.517965 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06deefa0-e424-4c78-9516-0bc6f42d1d55 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:29:13.519395 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 737/1491] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 5 14:29:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:29:13.524433 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e43bcb-f6d0-40e6-8953-0a12e1399ea3 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:29:13.524879 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e43bcb-f6d0-40e6-8953-0a12e1399ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:29:13.525010 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e43bcb-f6d0-40e6-8953-0a12e1399ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:29:13.590096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-f5e43bcb-f6d0-40e6-8953-0a12e1399ea3 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:29:13.591344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f5e43bcb-f6d0-40e6-8953-0a12e1399ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:29:13.686180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 755/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 5 14:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:29:13.976085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06deefa0-e424-4c78-9516-0bc6f42d1d55 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:29:13.977623 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-06deefa0-e424-4c78-9516-0bc6f42d1d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:29:14.081818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06deefa0-e424-4c78-9516-0bc6f42d1d55 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:29:14.155450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5e2b0-c571-41d3-a773-9227474714e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:29:14.155986 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5e2b0-c571-41d3-a773-9227474714e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:29:14.156262 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5e2b0-c571-41d3-a773-9227474714e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:29:14.156530 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5e2b0-c571-41d3-a773-9227474714e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:29:14.158683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ffb5e2b0-c571-41d3-a773-9227474714e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:29:14.167020 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:29:14.263693 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5e2b0-c571-41d3-a773-9227474714e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:29:14.265118 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5e2b0-c571-41d3-a773-9227474714e6 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:29:14.283909 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 738/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:29:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:29:23.684420 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=82 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:29:24.281158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=50 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:30:36.745766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d89ef-19a7-443e-9b2e-ceb695286429 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:30:36.746329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d89ef-19a7-443e-9b2e-ceb695286429 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:30:36.746329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d89ef-19a7-443e-9b2e-ceb695286429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:30:36.820432 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-578d89ef-19a7-443e-9b2e-ceb695286429 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:30:36.821632 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-578d89ef-19a7-443e-9b2e-ceb695286429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:30:36.917917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 756/1494] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:30:36.926025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708d7c9-cbf6-4ddb-bbf3-c34a86f87b3d None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:30:36.926533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708d7c9-cbf6-4ddb-bbf3-c34a86f87b3d None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:30:36.926638 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708d7c9-cbf6-4ddb-bbf3-c34a86f87b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:30:37.001558 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-c708d7c9-cbf6-4ddb-bbf3-c34a86f87b3d None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:30:37.002733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c708d7c9-cbf6-4ddb-bbf3-c34a86f87b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:30:37.098600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 739/1495] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:30:37.103321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d89ef-19a7-443e-9b2e-ceb695286429 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:30:37.104519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-578d89ef-19a7-443e-9b2e-ceb695286429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:30:37.115713 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d89ef-19a7-443e-9b2e-ceb695286429 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:30:37.176387 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:30:37.210069 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ebf46-8415-4a6b-8649-06d49f6dc9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:30:37.210673 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ebf46-8415-4a6b-8649-06d49f6dc9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:30:37.210978 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ebf46-8415-4a6b-8649-06d49f6dc9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:30:37.211247 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ebf46-8415-4a6b-8649-06d49f6dc9b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:30:37.211881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ebf46-8415-4a6b-8649-06d49f6dc9b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:30:37.212178 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ebf46-8415-4a6b-8649-06d49f6dc9b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:30:37.213191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: WARNING py.warnings [None req-8e7ebf46-8415-4a6b-8649-06d49f6dc9b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 05 14:30:37.213191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: warnings.warn(msg) Sep 05 14:30:37.213191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: Sep 05 14:30:37.214075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ebf46-8415-4a6b-8649-06d49f6dc9b3 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:30:37.225164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 757/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:30:37.233499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708d7c9-cbf6-4ddb-bbf3-c34a86f87b3d None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:30:37.242948 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708d7c9-cbf6-4ddb-bbf3-c34a86f87b3d None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:30:37.303476 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:30:37.337313 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e2d0d-52d6-429c-8d32-fe15ade05817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:30:37.337918 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e2d0d-52d6-429c-8d32-fe15ade05817 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:30:37.338199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e2d0d-52d6-429c-8d32-fe15ade05817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:30:37.338470 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e2d0d-52d6-429c-8d32-fe15ade05817 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:30:37.339142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e2d0d-52d6-429c-8d32-fe15ade05817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:30:37.340868 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: WARNING py.warnings [None req-6f3e2d0d-52d6-429c-8d32-fe15ade05817 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 14:30:37.340868 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: warnings.warn(msg) Sep 05 14:30:37.340868 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: Sep 05 14:30:37.342097 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e2d0d-52d6-429c-8d32-fe15ade05817 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:30:37.507066 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 740/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:30:37 2022] POST /identity/v3/projects => generated 379 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:30:37.511281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ebf46-8415-4a6b-8649-06d49f6dc9b3 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:30:37.512817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8e7ebf46-8415-4a6b-8649-06d49f6dc9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:30:37.544937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ebf46-8415-4a6b-8649-06d49f6dc9b3 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:30:37.639104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32431d2-49e4-459c-a37e-b0f3b5268c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:30:37.639660 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32431d2-49e4-459c-a37e-b0f3b5268c5b admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:30:37.639940 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32431d2-49e4-459c-a37e-b0f3b5268c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:30:37.640211 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32431d2-49e4-459c-a37e-b0f3b5268c5b admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:30:37.640835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32431d2-49e4-459c-a37e-b0f3b5268c5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:30:37.641841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: WARNING py.warnings [None req-a32431d2-49e4-459c-a37e-b0f3b5268c5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 14:30:37.641841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: warnings.warn(msg) Sep 05 14:30:37.641841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: Sep 05 14:30:37.643093 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32431d2-49e4-459c-a37e-b0f3b5268c5b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:30:37.801940 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 758/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:30:37 2022] POST /identity/v3/users => generated 290 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:30:37.806078 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e2d0d-52d6-429c-8d32-fe15ade05817 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:30:37.807641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-6f3e2d0d-52d6-429c-8d32-fe15ade05817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:30:37.869631 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e2d0d-52d6-429c-8d32-fe15ade05817 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:30:37.951004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:30:37.961914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbdd64-3011-4d7a-865a-c94327c24672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:30:37.962520 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbdd64-3011-4d7a-865a-c94327c24672 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:30:37.962816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbdd64-3011-4d7a-865a-c94327c24672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:30:37.963093 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbdd64-3011-4d7a-865a-c94327c24672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:30:37.963731 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becbdd64-3011-4d7a-865a-c94327c24672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:30:37.964744 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: WARNING py.warnings [None req-becbdd64-3011-4d7a-865a-c94327c24672 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 05 14:30:37.964744 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: warnings.warn(msg) Sep 05 14:30:37.964744 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: Sep 05 14:30:37.965667 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becbdd64-3011-4d7a-865a-c94327c24672 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:30:37.977708 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 741/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:30:37.982191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32431d2-49e4-459c-a37e-b0f3b5268c5b admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:30:37.983277 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a32431d2-49e4-459c-a37e-b0f3b5268c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:30:37.994845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32431d2-49e4-459c-a37e-b0f3b5268c5b admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:30:38.020607 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:30:38.090657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be403e-1f67-4ab1-9983-98ee32562ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:30:38.091498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be403e-1f67-4ab1-9983-98ee32562ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:30:38.091881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be403e-1f67-4ab1-9983-98ee32562ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:30:38.092273 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be403e-1f67-4ab1-9983-98ee32562ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:30:38.093593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be403e-1f67-4ab1-9983-98ee32562ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:30:38.094656 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: WARNING py.warnings [None req-90be403e-1f67-4ab1-9983-98ee32562ca5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 05 14:30:38.094656 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: warnings.warn(msg) Sep 05 14:30:38.094656 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: Sep 05 14:30:38.095604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be403e-1f67-4ab1-9983-98ee32562ca5 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:30:38.108912 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 759/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:30:38.113142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbdd64-3011-4d7a-865a-c94327c24672 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:30:38.114660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-becbdd64-3011-4d7a-865a-c94327c24672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:30:38.148076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbdd64-3011-4d7a-865a-c94327c24672 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:30:38.242042 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e4be5-6a84-4b27-8b31-3edf0bc7d1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:30:38.242702 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e4be5-6a84-4b27-8b31-3edf0bc7d1b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:30:38.243020 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e4be5-6a84-4b27-8b31-3edf0bc7d1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:30:38.243296 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e4be5-6a84-4b27-8b31-3edf0bc7d1b9 admin admin] PATH_INFO: `/v3/projects/ce31acc12c86400e91c11e7968e5da6a/users/13a4f0ef120b4abaa24108d14710ca29/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:30:38.265855 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e4be5-6a84-4b27-8b31-3edf0bc7d1b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce31acc12c86400e91c11e7968e5da6a, user_id=13a4f0ef120b4abaa24108d14710ca29, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:30:38.266916 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: WARNING py.warnings [None req-dd8e4be5-6a84-4b27-8b31-3edf0bc7d1b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 14:30:38.266916 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: warnings.warn(msg) Sep 05 14:30:38.266916 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: Sep 05 14:30:38.268966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e4be5-6a84-4b27-8b31-3edf0bc7d1b9 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:30:38.367609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 742/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:30:38 2022] PUT /identity/v3/projects/ce31acc12c86400e91c11e7968e5da6a/users/13a4f0ef120b4abaa24108d14710ca29/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:30:38.380539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51462e9-421e-4873-a109-fded30762194 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:30:38.380821 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51462e9-421e-4873-a109-fded30762194 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:30:38.381092 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51462e9-421e-4873-a109-fded30762194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:30:38.437453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-e51462e9-421e-4873-a109-fded30762194 None None] MFA Rules not processed for user `13a4f0ef120b4abaa24108d14710ca29`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:30:38.438681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e51462e9-421e-4873-a109-fded30762194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:30:38.517400 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 760/1502] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:30:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:30:38.530279 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d4fe4-4c46-49f4-b568-4fe5116c6da5 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:30:38.530570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d4fe4-4c46-49f4-b568-4fe5116c6da5 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:30:38.530884 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d4fe4-4c46-49f4-b568-4fe5116c6da5 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:30:38.532219 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 743/1503] 10.222.0.15 () {54 vars in 964 bytes} [Mon Sep 5 14:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:30:38.537124 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b783a-006b-41ec-a8a8-b7549de3582e None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:30:38.537415 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b783a-006b-41ec-a8a8-b7549de3582e None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:30:38.537693 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b783a-006b-41ec-a8a8-b7549de3582e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:30:38.569180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:30:38.613532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-5c1b783a-006b-41ec-a8a8-b7549de3582e None None] MFA Rules not processed for user `e49ac59302cb42f7942954d3859d3675`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:30:38.614738 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5c1b783a-006b-41ec-a8a8-b7549de3582e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:30:38.688971 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 761/1504] 10.222.0.15 () {58 vars in 1061 bytes} [Mon Sep 5 14:30:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:30:38.696998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d4fe4-4c46-49f4-b568-4fe5116c6da5 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:30:38.698497 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-6a2d4fe4-4c46-49f4-b568-4fe5116c6da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:30:38.749343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:30:38.802590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d4fe4-4c46-49f4-b568-4fe5116c6da5 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:30:38.877102 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bfd695-0e18-4a2d-9d58-c33d77b44055 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:30:38.877703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfd695-0e18-4a2d-9d58-c33d77b44055 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:30:38.877978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfd695-0e18-4a2d-9d58-c33d77b44055 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:30:38.878247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfd695-0e18-4a2d-9d58-c33d77b44055 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:30:38.879863 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b1bfd695-0e18-4a2d-9d58-c33d77b44055 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:30:38.975771 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bfd695-0e18-4a2d-9d58-c33d77b44055 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:30:38.977218 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bfd695-0e18-4a2d-9d58-c33d77b44055 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:30:38.980771 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:30:38.995359 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 744/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:30:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:30:39.040503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c77575-0bb6-4907-adb0-f5ee337d4217 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:30:39.040932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c77575-0bb6-4907-adb0-f5ee337d4217 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:30:39.041040 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c77575-0bb6-4907-adb0-f5ee337d4217 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:30:39.042332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 762/1506] 10.222.0.15 () {52 vars in 830 bytes} [Mon Sep 5 14:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:30:39.045893 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bfd695-0e18-4a2d-9d58-c33d77b44055 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:30:39.047148 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b1bfd695-0e18-4a2d-9d58-c33d77b44055 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:30:39.144571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bfd695-0e18-4a2d-9d58-c33d77b44055 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:30:39.237630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b59d9-becb-402c-b9bc-2870df4e8f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:30:39.238249 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b59d9-becb-402c-b9bc-2870df4e8f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:30:39.238527 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b59d9-becb-402c-b9bc-2870df4e8f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:30:39.238818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b59d9-becb-402c-b9bc-2870df4e8f67 admin admin] PATH_INFO: `/v3/users/13a4f0ef120b4abaa24108d14710ca29` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:30:39.250731 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b59d9-becb-402c-b9bc-2870df4e8f67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13a4f0ef120b4abaa24108d14710ca29)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:30:39.251792 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: WARNING py.warnings [None req-d21b59d9-becb-402c-b9bc-2870df4e8f67 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 14:30:39.251792 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: warnings.warn(msg) Sep 05 14:30:39.251792 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: Sep 05 14:30:39.252994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b59d9-becb-402c-b9bc-2870df4e8f67 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:30:39.262704 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:30:39.436826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-d21b59d9-becb-402c-b9bc-2870df4e8f67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13a4f0ef120b4abaa24108d14710ca29'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:30:39.497352 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-d21b59d9-becb-402c-b9bc-2870df4e8f67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13a4f0ef120b4abaa24108d14710ca29'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:30:39.497906 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-d21b59d9-becb-402c-b9bc-2870df4e8f67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13a4f0ef120b4abaa24108d14710ca29'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:30:39.521696 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-d21b59d9-becb-402c-b9bc-2870df4e8f67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13a4f0ef120b4abaa24108d14710ca29'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:30:39.535241 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-d21b59d9-becb-402c-b9bc-2870df4e8f67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13a4f0ef120b4abaa24108d14710ca29'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:30:39.540705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 745/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:30:39 2022] DELETE /identity/v3/users/13a4f0ef120b4abaa24108d14710ca29 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:30:39.545102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cf418-b97e-4d49-9a52-273d57d4dd92 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:30:39.545391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cf418-b97e-4d49-9a52-273d57d4dd92 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:30:39.545690 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cf418-b97e-4d49-9a52-273d57d4dd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:30:39.603496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-4d7cf418-b97e-4d49-9a52-273d57d4dd92 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:30:39.604708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-4d7cf418-b97e-4d49-9a52-273d57d4dd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:30:39.679267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:30:39.694146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 763/1508] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:30:39.701397 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b59d9-becb-402c-b9bc-2870df4e8f67 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:30:39.702621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d21b59d9-becb-402c-b9bc-2870df4e8f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:30:39.803967 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b59d9-becb-402c-b9bc-2870df4e8f67 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:30:39.857780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:30:39.881462 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5a14f-8386-43a3-bc47-f107c9f13e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:30:39.882181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5a14f-8386-43a3-bc47-f107c9f13e65 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:30:39.882503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5a14f-8386-43a3-bc47-f107c9f13e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:30:39.882776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5a14f-8386-43a3-bc47-f107c9f13e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:30:39.884400 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-82e5a14f-8386-43a3-bc47-f107c9f13e65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:30:39.997287 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5a14f-8386-43a3-bc47-f107c9f13e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:30:39.998745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5a14f-8386-43a3-bc47-f107c9f13e65 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:30:40.018455 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 746/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:30:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:30:40.688097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7cf418-b97e-4d49-9a52-273d57d4dd92 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:30:40.689328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-4d7cf418-b97e-4d49-9a52-273d57d4dd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:30:40.701458 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7cf418-b97e-4d49-9a52-273d57d4dd92 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:30:40.807368 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6a95a-bac2-4b56-9a84-5768fef6cb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:30:40.808020 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6a95a-bac2-4b56-9a84-5768fef6cb42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:30:40.808297 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6a95a-bac2-4b56-9a84-5768fef6cb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:30:40.808567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6a95a-bac2-4b56-9a84-5768fef6cb42 admin admin] PATH_INFO: `/v3/projects/ce31acc12c86400e91c11e7968e5da6a` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:30:40.818919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6a95a-bac2-4b56-9a84-5768fef6cb42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce31acc12c86400e91c11e7968e5da6a)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:30:40.819933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: WARNING py.warnings [None req-1cf6a95a-bac2-4b56-9a84-5768fef6cb42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 14:30:40.819933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: warnings.warn(msg) Sep 05 14:30:40.819933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: Sep 05 14:30:40.821127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6a95a-bac2-4b56-9a84-5768fef6cb42 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:30:40.942228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-1cf6a95a-bac2-4b56-9a84-5768fef6cb42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce31acc12c86400e91c11e7968e5da6a', 'request_id': 'req-1cf6a95a-bac2-4b56-9a84-5768fef6cb42'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:30:40.955109 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-1cf6a95a-bac2-4b56-9a84-5768fef6cb42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce31acc12c86400e91c11e7968e5da6a', 'request_id': 'req-1cf6a95a-bac2-4b56-9a84-5768fef6cb42'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:30:41.005718 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-1cf6a95a-bac2-4b56-9a84-5768fef6cb42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce31acc12c86400e91c11e7968e5da6a', 'request_id': 'req-1cf6a95a-bac2-4b56-9a84-5768fef6cb42'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:30:41.028554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-1cf6a95a-bac2-4b56-9a84-5768fef6cb42 admin admin] The token cache is being invalidate because project ce31acc12c86400e91c11e7968e5da6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:30:41.028847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-1cf6a95a-bac2-4b56-9a84-5768fef6cb42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:30:41.029367 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-1cf6a95a-bac2-4b56-9a84-5768fef6cb42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:30:41.030675 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 764/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:30:40 2022] DELETE /identity/v3/projects/ce31acc12c86400e91c11e7968e5da6a => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:30:50.016023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=62 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:30:51.026532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:32:13.731168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94251334-6bbd-4817-86c5-e225c002a5d6 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:13.731768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94251334-6bbd-4817-86c5-e225c002a5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:13.731768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94251334-6bbd-4817-86c5-e225c002a5d6 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:13.733067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 747/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 14:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:32:13.738445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8c88d-f595-4d9c-9630-4e8a0fc49ec3 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:13.738914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8c88d-f595-4d9c-9630-4e8a0fc49ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:13.739025 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8c88d-f595-4d9c-9630-4e8a0fc49ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:13.818460 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-3fb8c88d-f595-4d9c-9630-4e8a0fc49ec3 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:32:13.819705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-3fb8c88d-f595-4d9c-9630-4e8a0fc49ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:32:13.924970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 765/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 14:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:32:14.080189 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94251334-6bbd-4817-86c5-e225c002a5d6 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:32:14.087144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-94251334-6bbd-4817-86c5-e225c002a5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:32:14.202232 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94251334-6bbd-4817-86c5-e225c002a5d6 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:32:14.208566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc881523-1781-428a-b617-7a26ef4b13ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:32:14.214023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc881523-1781-428a-b617-7a26ef4b13ad service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:14.214023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc881523-1781-428a-b617-7a26ef4b13ad service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:14.214023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc881523-1781-428a-b617-7a26ef4b13ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:14.218034 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc881523-1781-428a-b617-7a26ef4b13ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:32:14.219682 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc881523-1781-428a-b617-7a26ef4b13ad service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:32:14.229552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 748/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:32:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:32:15.051137 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c97559-223c-4c9e-9650-7dde62849389 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:15.051706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c97559-223c-4c9e-9650-7dde62849389 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:15.051706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c97559-223c-4c9e-9650-7dde62849389 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:15.053013 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 766/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 14:32:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:32:15.058025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2812d-6ee6-43c4-bfb3-868b09442cd8 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:15.058420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2812d-6ee6-43c4-bfb3-868b09442cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:15.058581 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2812d-6ee6-43c4-bfb3-868b09442cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:15.078440 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-93b2812d-6ee6-43c4-bfb3-868b09442cd8 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:32:15.079669 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-93b2812d-6ee6-43c4-bfb3-868b09442cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:32:15.087822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 749/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 14:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:32:15.237792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c97559-223c-4c9e-9650-7dde62849389 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:32:15.252558 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c97559-223c-4c9e-9650-7dde62849389 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:32:15.258592 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3a0c6-68a3-4ffe-a874-48abfa0b0d8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:32:15.259225 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3a0c6-68a3-4ffe-a874-48abfa0b0d8d service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:15.259509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3a0c6-68a3-4ffe-a874-48abfa0b0d8d service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:15.259784 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3a0c6-68a3-4ffe-a874-48abfa0b0d8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:15.267709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3a0c6-68a3-4ffe-a874-48abfa0b0d8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:32:15.269318 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3a0c6-68a3-4ffe-a874-48abfa0b0d8d service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:32:15.278500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 767/1516] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:32:15.357617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2c7b8-7ca3-4e31-98e3-4e09dbac5850 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:15.357927 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2c7b8-7ca3-4e31-98e3-4e09dbac5850 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:15.358286 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2c7b8-7ca3-4e31-98e3-4e09dbac5850 None None] PATH_INFO: `/v3` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:15.359679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 750/1517] 10.222.0.50 () {54 vars in 925 bytes} [Mon Sep 5 14:32:15 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:32:15.366441 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed2096-c812-43f5-9481-91675c293dbe None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:15.366824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed2096-c812-43f5-9481-91675c293dbe None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:15.367171 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed2096-c812-43f5-9481-91675c293dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:15.428908 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-e9ed2096-c812-43f5-9481-91675c293dbe None None] MFA Rules not processed for user `267825dfb0324be0910203cd5abab377`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:32:15.430120 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e9ed2096-c812-43f5-9481-91675c293dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:32:15.446555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:32:15.502017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 768/1518] 10.222.0.50 () {58 vars in 1015 bytes} [Mon Sep 5 14:32:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:32:24.683428 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2332f-9b81-46bd-a144-0dca450acfe6 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:24.684059 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2332f-9b81-46bd-a144-0dca450acfe6 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:24.684059 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2332f-9b81-46bd-a144-0dca450acfe6 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:24.685235 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 751/1519] 10.222.0.15 () {54 vars in 961 bytes} [Mon Sep 5 14:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:32:24.690494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca0159-f22a-4904-b409-7c7a1cc0eeac None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:24.690976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca0159-f22a-4904-b409-7c7a1cc0eeac None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:24.691231 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca0159-f22a-4904-b409-7c7a1cc0eeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:24.710976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-7fca0159-f22a-4904-b409-7c7a1cc0eeac None None] MFA Rules not processed for user `267825dfb0324be0910203cd5abab377`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:32:24.712170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7fca0159-f22a-4904-b409-7c7a1cc0eeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:32:24.720624 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 769/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Sep 5 14:32:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:32:24.728126 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2332f-9b81-46bd-a144-0dca450acfe6 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:32:24.741700 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2332f-9b81-46bd-a144-0dca450acfe6 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:32:24.746609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbe48b-3c94-4ea9-a6cc-c463a81ca9cf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:32:24.747225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbe48b-3c94-4ea9-a6cc-c463a81ca9cf service placement] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:24.747511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbe48b-3c94-4ea9-a6cc-c463a81ca9cf service placement] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:24.747848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbe48b-3c94-4ea9-a6cc-c463a81ca9cf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:24.756439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efbe48b-3c94-4ea9-a6cc-c463a81ca9cf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:32:24.761825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efbe48b-3c94-4ea9-a6cc-c463a81ca9cf service placement] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:32:24.767447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 752/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:32:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:32:26.524651 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa272303-7d1b-4d68-b59f-1f709c3d8417 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:26.525214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa272303-7d1b-4d68-b59f-1f709c3d8417 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:26.525214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa272303-7d1b-4d68-b59f-1f709c3d8417 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:26.526555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 770/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 14:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:32:26.531292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6b806-8bc2-494c-afdb-6b1d45fa86f4 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:26.531714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6b806-8bc2-494c-afdb-6b1d45fa86f4 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:26.531832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6b806-8bc2-494c-afdb-6b1d45fa86f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:26.552064 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-77d6b806-8bc2-494c-afdb-6b1d45fa86f4 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:32:26.553253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-77d6b806-8bc2-494c-afdb-6b1d45fa86f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:32:26.560924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 753/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 14:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:32:26.712696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa272303-7d1b-4d68-b59f-1f709c3d8417 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:32:26.727645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa272303-7d1b-4d68-b59f-1f709c3d8417 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:32:26.733114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530bb69-9d62-491a-b9b8-c07a8486baa9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:32:26.733710 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530bb69-9d62-491a-b9b8-c07a8486baa9 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:26.733995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530bb69-9d62-491a-b9b8-c07a8486baa9 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:26.734269 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530bb69-9d62-491a-b9b8-c07a8486baa9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:26.742931 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530bb69-9d62-491a-b9b8-c07a8486baa9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:32:26.744510 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530bb69-9d62-491a-b9b8-c07a8486baa9 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:32:26.753984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 771/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:32:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:32:27.587903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdb5d8-0971-43f2-a77c-bf1ab04f6667 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:27.588478 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdb5d8-0971-43f2-a77c-bf1ab04f6667 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:27.588478 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdb5d8-0971-43f2-a77c-bf1ab04f6667 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:27.589828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 754/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 14:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:32:27.594799 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10ebfa-eb5e-4a42-b04b-0903af15e3c4 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:27.595200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10ebfa-eb5e-4a42-b04b-0903af15e3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:27.595351 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10ebfa-eb5e-4a42-b04b-0903af15e3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:27.614288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-2d10ebfa-eb5e-4a42-b04b-0903af15e3c4 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:32:27.615503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-2d10ebfa-eb5e-4a42-b04b-0903af15e3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:32:27.623231 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 772/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 14:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:32:27.774894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdb5d8-0971-43f2-a77c-bf1ab04f6667 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:32:27.789000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdb5d8-0971-43f2-a77c-bf1ab04f6667 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:32:27.794510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83068fc7-5632-4ea0-831b-0c275a0fc79b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:32:27.797942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068fc7-5632-4ea0-831b-0c275a0fc79b service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:27.797942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068fc7-5632-4ea0-831b-0c275a0fc79b service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:27.797942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068fc7-5632-4ea0-831b-0c275a0fc79b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:27.804703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83068fc7-5632-4ea0-831b-0c275a0fc79b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:32:27.806289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83068fc7-5632-4ea0-831b-0c275a0fc79b service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:32:27.815671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 755/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:32:31.547986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd67ce-da23-4646-b10d-1b3b3f35410e None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:31.548540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd67ce-da23-4646-b10d-1b3b3f35410e None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:31.548653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd67ce-da23-4646-b10d-1b3b3f35410e None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:31.550117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 773/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 14:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:32:31.555850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dab7f2-75f9-4779-b53d-125864212750 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:31.556303 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dab7f2-75f9-4779-b53d-125864212750 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:31.556401 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dab7f2-75f9-4779-b53d-125864212750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:31.576949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-10dab7f2-75f9-4779-b53d-125864212750 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:32:31.578141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-10dab7f2-75f9-4779-b53d-125864212750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:32:31.586201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 756/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 14:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:32:31.870172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd67ce-da23-4646-b10d-1b3b3f35410e None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:32:31.871912 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-effd67ce-da23-4646-b10d-1b3b3f35410e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:32:31.948721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd67ce-da23-4646-b10d-1b3b3f35410e None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:32:31.952216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32625db2-854f-464b-a5f7-7b3d96fb85ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:32:31.952815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32625db2-854f-464b-a5f7-7b3d96fb85ae service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:31.953096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32625db2-854f-464b-a5f7-7b3d96fb85ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:31.953366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32625db2-854f-464b-a5f7-7b3d96fb85ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:31.962267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32625db2-854f-464b-a5f7-7b3d96fb85ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:32:31.963715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32625db2-854f-464b-a5f7-7b3d96fb85ae service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:32:31.973809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 774/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:32:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:32:33.865215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc307cf-23df-43d9-84b8-62b5acbe5e67 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:33.865747 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc307cf-23df-43d9-84b8-62b5acbe5e67 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:33.865747 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc307cf-23df-43d9-84b8-62b5acbe5e67 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:33.867161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 757/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 14:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:32:33.872454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db501f0e-b087-42a5-b965-8810ec4173ac None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:33.875077 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db501f0e-b087-42a5-b965-8810ec4173ac None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:33.875077 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db501f0e-b087-42a5-b965-8810ec4173ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:33.892382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-db501f0e-b087-42a5-b965-8810ec4173ac None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:32:33.893593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-db501f0e-b087-42a5-b965-8810ec4173ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:32:33.901871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 775/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 14:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:32:34.186352 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc307cf-23df-43d9-84b8-62b5acbe5e67 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:32:34.188178 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-1cc307cf-23df-43d9-84b8-62b5acbe5e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:32:34.201206 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc307cf-23df-43d9-84b8-62b5acbe5e67 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:32:34.204658 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1e85c-8f47-4f1d-b6e6-f5d67c87c108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:32:34.205252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1e85c-8f47-4f1d-b6e6-f5d67c87c108 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:34.205534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1e85c-8f47-4f1d-b6e6-f5d67c87c108 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:34.205806 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1e85c-8f47-4f1d-b6e6-f5d67c87c108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:34.214736 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1e85c-8f47-4f1d-b6e6-f5d67c87c108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:32:34.216202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1e85c-8f47-4f1d-b6e6-f5d67c87c108 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:32:34.226012 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 758/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:32:37.465868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7adcbe-30c1-492a-a9dc-679e2a43444d None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:37.466403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7adcbe-30c1-492a-a9dc-679e2a43444d None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:37.466403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7adcbe-30c1-492a-a9dc-679e2a43444d None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:37.467722 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 776/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 14:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:32:37.472726 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe88379-7ed1-434f-801c-277a1212b611 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:37.473165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe88379-7ed1-434f-801c-277a1212b611 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:37.473323 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe88379-7ed1-434f-801c-277a1212b611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:37.492589 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-bfe88379-7ed1-434f-801c-277a1212b611 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:32:37.493769 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-bfe88379-7ed1-434f-801c-277a1212b611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:32:37.501387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 759/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 14:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:32:38.056021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7adcbe-30c1-492a-a9dc-679e2a43444d None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:32:38.057756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7b7adcbe-30c1-492a-a9dc-679e2a43444d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:32:38.124512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:32:38.137213 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7adcbe-30c1-492a-a9dc-679e2a43444d None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:32:38.140775 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ca0f8-f8fd-4c47-8f3f-fe52d771653b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:32:38.141379 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ca0f8-f8fd-4c47-8f3f-fe52d771653b service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:38.141661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ca0f8-f8fd-4c47-8f3f-fe52d771653b service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:38.141936 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ca0f8-f8fd-4c47-8f3f-fe52d771653b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:38.151150 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ca0f8-f8fd-4c47-8f3f-fe52d771653b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:32:38.152577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ca0f8-f8fd-4c47-8f3f-fe52d771653b service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:32:38.163020 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 777/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:32:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:32:40.072515 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef91605-0bb4-4925-89a4-6916d8710a6f None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:40.073114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef91605-0bb4-4925-89a4-6916d8710a6f None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:40.073229 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef91605-0bb4-4925-89a4-6916d8710a6f None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:40.074614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 760/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 14:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:32:40.080233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12623ef1-ed03-48b7-9806-aad1e200f0bd None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:40.080674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12623ef1-ed03-48b7-9806-aad1e200f0bd None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:40.080797 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12623ef1-ed03-48b7-9806-aad1e200f0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:40.101693 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-12623ef1-ed03-48b7-9806-aad1e200f0bd None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:32:40.103001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-12623ef1-ed03-48b7-9806-aad1e200f0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:32:40.111113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 778/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 14:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:32:40.655795 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef91605-0bb4-4925-89a4-6916d8710a6f None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:32:40.657492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-eef91605-0bb4-4925-89a4-6916d8710a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:32:40.665071 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=84 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:32:40.670764 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef91605-0bb4-4925-89a4-6916d8710a6f None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:32:40.676219 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0e24e-0978-464a-8e65-cebdd4f8c0b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:32:40.676219 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0e24e-0978-464a-8e65-cebdd4f8c0b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:32:40.676219 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0e24e-0978-464a-8e65-cebdd4f8c0b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:32:40.676219 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0e24e-0978-464a-8e65-cebdd4f8c0b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:32:40.684917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0e24e-0978-464a-8e65-cebdd4f8c0b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:32:40.686333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0e24e-0978-464a-8e65-cebdd4f8c0b2 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:32:40.696447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 761/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:32:50.096530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=11 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:32:50.697408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:06.842885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d4f43-b650-4aec-82cf-5ba0b45f297b None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:06.843541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d4f43-b650-4aec-82cf-5ba0b45f297b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:06.843541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d4f43-b650-4aec-82cf-5ba0b45f297b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:06.873043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-c07d4f43-b650-4aec-82cf-5ba0b45f297b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:06.874967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c07d4f43-b650-4aec-82cf-5ba0b45f297b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:06.888049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 779/1540] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:06.897456 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2697a09-8a2d-4c23-9ba8-444eb8dc700c None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:06.898811 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2697a09-8a2d-4c23-9ba8-444eb8dc700c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:06.899401 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2697a09-8a2d-4c23-9ba8-444eb8dc700c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:06.926235 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-a2697a09-8a2d-4c23-9ba8-444eb8dc700c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:06.927897 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a2697a09-8a2d-4c23-9ba8-444eb8dc700c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:06.937270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 762/1541] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:06.941787 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d4f43-b650-4aec-82cf-5ba0b45f297b None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:06.953100 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d4f43-b650-4aec-82cf-5ba0b45f297b None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:06.960002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf93a6-bcdb-4ffe-9855-83894bf41d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:06.960904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf93a6-bcdb-4ffe-9855-83894bf41d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:06.961396 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf93a6-bcdb-4ffe-9855-83894bf41d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:06.961900 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf93a6-bcdb-4ffe-9855-83894bf41d7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:06.963057 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf93a6-bcdb-4ffe-9855-83894bf41d7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:06.963586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf93a6-bcdb-4ffe-9855-83894bf41d7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:06.965541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf93a6-bcdb-4ffe-9855-83894bf41d7a admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:06.980711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 780/1542] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:34:06.988840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2697a09-8a2d-4c23-9ba8-444eb8dc700c None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:06.999528 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2697a09-8a2d-4c23-9ba8-444eb8dc700c None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:07.006058 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831f1fd-1cef-4eb6-a682-b9e69cfde200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:07.007084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831f1fd-1cef-4eb6-a682-b9e69cfde200 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.007602 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831f1fd-1cef-4eb6-a682-b9e69cfde200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.008107 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831f1fd-1cef-4eb6-a682-b9e69cfde200 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.008970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831f1fd-1cef-4eb6-a682-b9e69cfde200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:07.011519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831f1fd-1cef-4eb6-a682-b9e69cfde200 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:07.134677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 763/1543] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:06 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:07.139011 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf93a6-bcdb-4ffe-9855-83894bf41d7a admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:07.151496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf93a6-bcdb-4ffe-9855-83894bf41d7a admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:07.224468 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88552446-a006-400e-9d0b-8917a8934f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:07.225674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88552446-a006-400e-9d0b-8917a8934f13 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.226233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88552446-a006-400e-9d0b-8917a8934f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.226743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88552446-a006-400e-9d0b-8917a8934f13 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.227651 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88552446-a006-400e-9d0b-8917a8934f13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:07.230290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88552446-a006-400e-9d0b-8917a8934f13 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:07.262405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29d904-1fa7-4f42-8f38-c9911e4ba18a None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.263132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29d904-1fa7-4f42-8f38-c9911e4ba18a None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.263644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29d904-1fa7-4f42-8f38-c9911e4ba18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.293143 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-1d29d904-1fa7-4f42-8f38-c9911e4ba18a None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:07.294864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-1d29d904-1fa7-4f42-8f38-c9911e4ba18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:07.311618 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 764/1544] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:34:07.313510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68996e-cf98-4345-a995-ffc852d68264 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.313809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68996e-cf98-4345-a995-ffc852d68264 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.314091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68996e-cf98-4345-a995-ffc852d68264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.333530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 781/1545] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:07 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:07.335094 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c695845-0223-45ac-b95d-8ca04c76779f None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.335390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c695845-0223-45ac-b95d-8ca04c76779f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.335672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c695845-0223-45ac-b95d-8ca04c76779f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.360404 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-af68996e-cf98-4345-a995-ffc852d68264 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:07.369017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-af68996e-cf98-4345-a995-ffc852d68264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:07.386699 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-3c695845-0223-45ac-b95d-8ca04c76779f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:07.390919 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 765/1546] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:07.391406 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-3c695845-0223-45ac-b95d-8ca04c76779f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:07.392519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68996e-cf98-4345-a995-ffc852d68264 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:07.412574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 782/1547] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:07.414220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68996e-cf98-4345-a995-ffc852d68264 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:07.415959 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06984f80-b5ee-477f-83b3-f727b1954161 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.416247 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06984f80-b5ee-477f-83b3-f727b1954161 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.416521 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06984f80-b5ee-477f-83b3-f727b1954161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.427469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73ec07-73da-4f45-a6c0-a43c3f137f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:07.428110 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73ec07-73da-4f45-a6c0-a43c3f137f10 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.428397 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73ec07-73da-4f45-a6c0-a43c3f137f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.428719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73ec07-73da-4f45-a6c0-a43c3f137f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.432259 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73ec07-73da-4f45-a6c0-a43c3f137f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:07.434658 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73ec07-73da-4f45-a6c0-a43c3f137f10 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:07.457979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 766/1548] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:07.460356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-06984f80-b5ee-477f-83b3-f727b1954161 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:07.461321 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73ec07-73da-4f45-a6c0-a43c3f137f10 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:07.464482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-06984f80-b5ee-477f-83b3-f727b1954161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:07.480211 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73ec07-73da-4f45-a6c0-a43c3f137f10 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:07.482340 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 783/1549] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:07.483965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06984f80-b5ee-477f-83b3-f727b1954161 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:07.501660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f275fba-4212-4cce-9524-1f0f25167302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:07.502295 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f275fba-4212-4cce-9524-1f0f25167302 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.503495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f275fba-4212-4cce-9524-1f0f25167302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.503926 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f275fba-4212-4cce-9524-1f0f25167302 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.504594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f275fba-4212-4cce-9524-1f0f25167302 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:07.504904 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f275fba-4212-4cce-9524-1f0f25167302 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:07.505946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: WARNING py.warnings [None req-8f275fba-4212-4cce-9524-1f0f25167302 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 05 14:34:07.505946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: warnings.warn(msg) Sep 05 14:34:07.505946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: Sep 05 14:34:07.507855 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f275fba-4212-4cce-9524-1f0f25167302 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:07.518614 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06984f80-b5ee-477f-83b3-f727b1954161 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:07.530517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b758c-b55d-47b6-a8c1-03a2bf9df973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:07.531817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b758c-b55d-47b6-a8c1-03a2bf9df973 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.531817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b758c-b55d-47b6-a8c1-03a2bf9df973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.532021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b758c-b55d-47b6-a8c1-03a2bf9df973 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.532621 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b758c-b55d-47b6-a8c1-03a2bf9df973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:07.535835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b758c-b55d-47b6-a8c1-03a2bf9df973 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:07.539033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 767/1550] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:34:07.539915 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f275fba-4212-4cce-9524-1f0f25167302 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:07.560899 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f275fba-4212-4cce-9524-1f0f25167302 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:07.561561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 784/1551] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:07.563044 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a46cf8-53fa-49ed-862d-90ccc0d81778 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.563826 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a46cf8-53fa-49ed-862d-90ccc0d81778 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.564112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a46cf8-53fa-49ed-862d-90ccc0d81778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.574627 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c0135-619d-4423-b4ab-ae96468d3f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:07.575337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c0135-619d-4423-b4ab-ae96468d3f6a admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.575628 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c0135-619d-4423-b4ab-ae96468d3f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.575907 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c0135-619d-4423-b4ab-ae96468d3f6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.578147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c0135-619d-4423-b4ab-ae96468d3f6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:07.578587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c0135-619d-4423-b4ab-ae96468d3f6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:07.580248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c0135-619d-4423-b4ab-ae96468d3f6a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:07.606251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-96a46cf8-53fa-49ed-862d-90ccc0d81778 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:07.609218 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 768/1552] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:34:07.610086 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c0135-619d-4423-b4ab-ae96468d3f6a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:07.611644 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-96a46cf8-53fa-49ed-862d-90ccc0d81778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:07.632288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 785/1553] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:07.633465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c0135-619d-4423-b4ab-ae96468d3f6a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:07.634412 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a46cf8-53fa-49ed-862d-90ccc0d81778 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:07.645051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904e926-e387-4747-9f2e-0973c407bee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:07.646276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904e926-e387-4747-9f2e-0973c407bee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.646591 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904e926-e387-4747-9f2e-0973c407bee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.646902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904e926-e387-4747-9f2e-0973c407bee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.648461 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904e926-e387-4747-9f2e-0973c407bee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:07.650708 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904e926-e387-4747-9f2e-0973c407bee4 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:07.657959 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a46cf8-53fa-49ed-862d-90ccc0d81778 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:07.668211 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af35c4-ba01-4531-868b-694a628c47f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:07.669214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af35c4-ba01-4531-868b-694a628c47f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.669710 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af35c4-ba01-4531-868b-694a628c47f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.670818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af35c4-ba01-4531-868b-694a628c47f9 admin admin] PATH_INFO: `/v3/projects/0f14cb621ef04aa3a17159e418d6c7aa/users/c078e655c9454316aa7afe13a6b565e7/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.689818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af35c4-ba01-4531-868b-694a628c47f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f14cb621ef04aa3a17159e418d6c7aa, user_id=c078e655c9454316aa7afe13a6b565e7, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:07.691817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: WARNING py.warnings [None req-50af35c4-ba01-4531-868b-694a628c47f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 14:34:07.691817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: warnings.warn(msg) Sep 05 14:34:07.691817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: Sep 05 14:34:07.693614 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af35c4-ba01-4531-868b-694a628c47f9 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:07.778426 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 769/1554] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:07 2022] POST /identity/v3/projects => generated 373 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:07.779409 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904e926-e387-4747-9f2e-0973c407bee4 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:07.800288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 786/1555] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:07 2022] PUT /identity/v3/projects/0f14cb621ef04aa3a17159e418d6c7aa/users/c078e655c9454316aa7afe13a6b565e7/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:07.806893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41168fa7-f3a2-4e21-85ef-7465e6ae9776 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.806893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41168fa7-f3a2-4e21-85ef-7465e6ae9776 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.806893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41168fa7-f3a2-4e21-85ef-7465e6ae9776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.821390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-41168fa7-f3a2-4e21-85ef-7465e6ae9776 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:07.822635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-41168fa7-f3a2-4e21-85ef-7465e6ae9776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:07.844295 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904e926-e387-4747-9f2e-0973c407bee4 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:07.881728 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 787/1556] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:07.882916 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f2090-3773-44e5-9613-3b0c3d08e710 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.883203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f2090-3773-44e5-9613-3b0c3d08e710 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.883476 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f2090-3773-44e5-9613-3b0c3d08e710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.898118 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:07.904057 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141304b-cb9b-4821-83ee-af9c760ea803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:07.904218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-873f2090-3773-44e5-9613-3b0c3d08e710 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:07.904672 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141304b-cb9b-4821-83ee-af9c760ea803 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.904993 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141304b-cb9b-4821-83ee-af9c760ea803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.905233 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141304b-cb9b-4821-83ee-af9c760ea803 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.905382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-873f2090-3773-44e5-9613-3b0c3d08e710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:07.905941 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141304b-cb9b-4821-83ee-af9c760ea803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:07.908029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141304b-cb9b-4821-83ee-af9c760ea803 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:07.913925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 788/1557] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:07.915125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8c7c4-076b-4522-8ed6-898e570dff6f None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.915407 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8c7c4-076b-4522-8ed6-898e570dff6f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.915678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8c7c4-076b-4522-8ed6-898e570dff6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.934679 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-59a8c7c4-076b-4522-8ed6-898e570dff6f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:07.935897 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-59a8c7c4-076b-4522-8ed6-898e570dff6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:07.944241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 789/1558] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:07.945389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d32e98-db93-4172-b97a-0ae6739a29c7 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.945653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d32e98-db93-4172-b97a-0ae6739a29c7 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.945923 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d32e98-db93-4172-b97a-0ae6739a29c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.953057 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:07.965867 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-09d32e98-db93-4172-b97a-0ae6739a29c7 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:07.967110 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-09d32e98-db93-4172-b97a-0ae6739a29c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:07.974899 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 790/1559] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:07.976044 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee37b48-49b5-40b1-8317-22d862918ddd None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:07.976331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee37b48-49b5-40b1-8317-22d862918ddd None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:07.976605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee37b48-49b5-40b1-8317-22d862918ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:07.985295 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 770/1560] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:07 2022] POST /identity/v3/projects => generated 405 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:07.985692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141304b-cb9b-4821-83ee-af9c760ea803 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:07.995556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-1ee37b48-49b5-40b1-8317-22d862918ddd None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:07.996736 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-1ee37b48-49b5-40b1-8317-22d862918ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:08.046198 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141304b-cb9b-4821-83ee-af9c760ea803 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.050952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46266ea8-f876-4564-9d1c-8454197ef81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.051563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46266ea8-f876-4564-9d1c-8454197ef81d admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.051847 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46266ea8-f876-4564-9d1c-8454197ef81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.052123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46266ea8-f876-4564-9d1c-8454197ef81d admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.052750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46266ea8-f876-4564-9d1c-8454197ef81d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.053777 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: WARNING py.warnings [None req-46266ea8-f876-4564-9d1c-8454197ef81d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 14:34:08.053777 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: warnings.warn(msg) Sep 05 14:34:08.053777 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: Sep 05 14:34:08.054826 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 791/1561] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:08.058986 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46266ea8-f876-4564-9d1c-8454197ef81d admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.059104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b1b2f-5ddc-4f12-a6ff-9ea7696158f8 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.059104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b1b2f-5ddc-4f12-a6ff-9ea7696158f8 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.059104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b1b2f-5ddc-4f12-a6ff-9ea7696158f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.079370 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-980b1b2f-5ddc-4f12-a6ff-9ea7696158f8 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:08.081137 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-980b1b2f-5ddc-4f12-a6ff-9ea7696158f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:08.089118 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 792/1562] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:08.089930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b1b2f-5ddc-4f12-a6ff-9ea7696158f8 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:08.101290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b1b2f-5ddc-4f12-a6ff-9ea7696158f8 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.105822 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750d8ee-8308-4b64-a6b0-6f185e986e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.106412 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750d8ee-8308-4b64-a6b0-6f185e986e3c admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.106692 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750d8ee-8308-4b64-a6b0-6f185e986e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.107026 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750d8ee-8308-4b64-a6b0-6f185e986e3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.107663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2750d8ee-8308-4b64-a6b0-6f185e986e3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:08.107959 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2750d8ee-8308-4b64-a6b0-6f185e986e3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.109534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2750d8ee-8308-4b64-a6b0-6f185e986e3c admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.121196 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 793/1563] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:34:08.122294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4491da0-5d37-4847-b4bd-63384337bbbf None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.122605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4491da0-5d37-4847-b4bd-63384337bbbf None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.122905 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4491da0-5d37-4847-b4bd-63384337bbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.142126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-e4491da0-5d37-4847-b4bd-63384337bbbf None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:08.142894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 771/1564] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:07 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:08.143320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e4491da0-5d37-4847-b4bd-63384337bbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:08.144213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9120237-f73f-4ee3-8fd9-ed95611e9fd0 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.144505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9120237-f73f-4ee3-8fd9-ed95611e9fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.144783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9120237-f73f-4ee3-8fd9-ed95611e9fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.151554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 794/1565] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:08.152659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90868c0-47c0-4781-a550-994cfdbe5f82 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.152885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90868c0-47c0-4781-a550-994cfdbe5f82 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.153155 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90868c0-47c0-4781-a550-994cfdbe5f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.168776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-f9120237-f73f-4ee3-8fd9-ed95611e9fd0 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:08.170007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f9120237-f73f-4ee3-8fd9-ed95611e9fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:08.172864 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-c90868c0-47c0-4781-a550-994cfdbe5f82 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:08.174017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c90868c0-47c0-4781-a550-994cfdbe5f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:08.178414 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 772/1566] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:08.179188 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9120237-f73f-4ee3-8fd9-ed95611e9fd0 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:08.182809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 795/1567] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:08.183980 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2afc24-47c6-4959-9a0e-9541840d03ba None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.184271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2afc24-47c6-4959-9a0e-9541840d03ba None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.184526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2afc24-47c6-4959-9a0e-9541840d03ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.189284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9120237-f73f-4ee3-8fd9-ed95611e9fd0 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.195088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50740b07-95b7-4062-8b0f-449d6f013607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.195870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50740b07-95b7-4062-8b0f-449d6f013607 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.196157 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50740b07-95b7-4062-8b0f-449d6f013607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.196431 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50740b07-95b7-4062-8b0f-449d6f013607 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.197059 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50740b07-95b7-4062-8b0f-449d6f013607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.199465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50740b07-95b7-4062-8b0f-449d6f013607 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.203517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-6c2afc24-47c6-4959-9a0e-9541840d03ba None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:08.204699 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-6c2afc24-47c6-4959-9a0e-9541840d03ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:08.215966 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 796/1568] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:08.216786 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2afc24-47c6-4959-9a0e-9541840d03ba None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:08.271415 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2afc24-47c6-4959-9a0e-9541840d03ba None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.275061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3943557-af7a-4dab-9ecf-e91d1ffb339c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.275714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3943557-af7a-4dab-9ecf-e91d1ffb339c service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.275996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3943557-af7a-4dab-9ecf-e91d1ffb339c service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.276267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3943557-af7a-4dab-9ecf-e91d1ffb339c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.285855 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3943557-af7a-4dab-9ecf-e91d1ffb339c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.287484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3943557-af7a-4dab-9ecf-e91d1ffb339c service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.291696 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 773/1569] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:08 2022] POST /identity/v3/users => generated 303 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:08.292490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50740b07-95b7-4062-8b0f-449d6f013607 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:08.300438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 797/1570] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:08.301303 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3943557-af7a-4dab-9ecf-e91d1ffb339c service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:08.302030 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50740b07-95b7-4062-8b0f-449d6f013607 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.306753 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984e6d8-2f67-40ee-aae8-06a189afc869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.307600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984e6d8-2f67-40ee-aae8-06a189afc869 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.307758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984e6d8-2f67-40ee-aae8-06a189afc869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.308018 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984e6d8-2f67-40ee-aae8-06a189afc869 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.308639 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5984e6d8-2f67-40ee-aae8-06a189afc869 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.310531 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5984e6d8-2f67-40ee-aae8-06a189afc869 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.311346 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3943557-af7a-4dab-9ecf-e91d1ffb339c service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.316674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec55311-f6ca-4a3c-aa16-b69621c614ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.317466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec55311-f6ca-4a3c-aa16-b69621c614ac admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.317896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec55311-f6ca-4a3c-aa16-b69621c614ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.318183 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec55311-f6ca-4a3c-aa16-b69621c614ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.318979 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec55311-f6ca-4a3c-aa16-b69621c614ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.320690 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec55311-f6ca-4a3c-aa16-b69621c614ac admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.331629 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 798/1571] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:08.332505 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec55311-f6ca-4a3c-aa16-b69621c614ac admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:08.342151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec55311-f6ca-4a3c-aa16-b69621c614ac admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.347316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fad63-9155-45bc-91ad-fefbab089e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.347914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fad63-9155-45bc-91ad-fefbab089e54 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.348193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fad63-9155-45bc-91ad-fefbab089e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.348459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fad63-9155-45bc-91ad-fefbab089e54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.349113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8fad63-9155-45bc-91ad-fefbab089e54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:08.349409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8fad63-9155-45bc-91ad-fefbab089e54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.350988 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8fad63-9155-45bc-91ad-fefbab089e54 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.362599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 799/1572] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:34:08.363391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fad63-9155-45bc-91ad-fefbab089e54 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:08.372565 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fad63-9155-45bc-91ad-fefbab089e54 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.377598 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8922c7-75b2-4ff4-86a1-8b847024ecef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.378266 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8922c7-75b2-4ff4-86a1-8b847024ecef admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.378618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8922c7-75b2-4ff4-86a1-8b847024ecef admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.379027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8922c7-75b2-4ff4-86a1-8b847024ecef admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.379753 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8922c7-75b2-4ff4-86a1-8b847024ecef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:08.380164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8922c7-75b2-4ff4-86a1-8b847024ecef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.381792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8922c7-75b2-4ff4-86a1-8b847024ecef admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.394469 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 800/1573] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:34:08.395319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8922c7-75b2-4ff4-86a1-8b847024ecef admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:08.405429 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8922c7-75b2-4ff4-86a1-8b847024ecef admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.411081 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c22ea-2604-41b4-980c-d5fbb10b47f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.415515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c22ea-2604-41b4-980c-d5fbb10b47f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.415515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c22ea-2604-41b4-980c-d5fbb10b47f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.415515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c22ea-2604-41b4-980c-d5fbb10b47f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.415515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c22ea-2604-41b4-980c-d5fbb10b47f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:08.415515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c22ea-2604-41b4-980c-d5fbb10b47f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.415515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c22ea-2604-41b4-980c-d5fbb10b47f9 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.418296 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 774/1574] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:08 2022] POST /identity/v3/projects => generated 389 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:08.419430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984e6d8-2f67-40ee-aae8-06a189afc869 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:08.429763 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 801/1575] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:34:08.430731 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c22ea-2604-41b4-980c-d5fbb10b47f9 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:08.442131 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c22ea-2604-41b4-980c-d5fbb10b47f9 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.447244 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:08.495677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984e6d8-2f67-40ee-aae8-06a189afc869 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.500448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384eeb02-f7f4-4efb-9810-0f447b3acef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.501064 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eeb02-f7f4-4efb-9810-0f447b3acef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.501339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eeb02-f7f4-4efb-9810-0f447b3acef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.501606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eeb02-f7f4-4efb-9810-0f447b3acef6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.501910 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04aec2-ae14-4288-8513-b58aead4108c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.502239 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384eeb02-f7f4-4efb-9810-0f447b3acef6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:08.503151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04aec2-ae14-4288-8513-b58aead4108c admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.503268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384eeb02-f7f4-4efb-9810-0f447b3acef6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.503567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04aec2-ae14-4288-8513-b58aead4108c admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.503900 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04aec2-ae14-4288-8513-b58aead4108c admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.504638 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04aec2-ae14-4288-8513-b58aead4108c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.504844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384eeb02-f7f4-4efb-9810-0f447b3acef6 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.506225 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04aec2-ae14-4288-8513-b58aead4108c admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.516124 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 802/1576] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:08.516620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 775/1577] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:34:08.517425 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384eeb02-f7f4-4efb-9810-0f447b3acef6 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:08.520737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04aec2-ae14-4288-8513-b58aead4108c admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:08.528624 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384eeb02-f7f4-4efb-9810-0f447b3acef6 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.530396 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04aec2-ae14-4288-8513-b58aead4108c admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.533108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c78b6c-1487-47b1-9802-c7045acee021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.533722 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c78b6c-1487-47b1-9802-c7045acee021 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.534056 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c78b6c-1487-47b1-9802-c7045acee021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.534267 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c78b6c-1487-47b1-9802-c7045acee021 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.534919 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c78b6c-1487-47b1-9802-c7045acee021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.536173 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82378b11-ad0c-4008-85b8-b49d88476719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.536760 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82378b11-ad0c-4008-85b8-b49d88476719 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.536879 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c78b6c-1487-47b1-9802-c7045acee021 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.537042 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82378b11-ad0c-4008-85b8-b49d88476719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.537317 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82378b11-ad0c-4008-85b8-b49d88476719 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.537952 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82378b11-ad0c-4008-85b8-b49d88476719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.539607 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82378b11-ad0c-4008-85b8-b49d88476719 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.549561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 803/1578] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:08.550405 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82378b11-ad0c-4008-85b8-b49d88476719 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:08.560040 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82378b11-ad0c-4008-85b8-b49d88476719 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.564551 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4252fb-51e2-4094-9103-69ae03b03932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.565664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4252fb-51e2-4094-9103-69ae03b03932 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.566017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4252fb-51e2-4094-9103-69ae03b03932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.566297 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4252fb-51e2-4094-9103-69ae03b03932 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.566946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4252fb-51e2-4094-9103-69ae03b03932 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.567942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: WARNING py.warnings [None req-4f4252fb-51e2-4094-9103-69ae03b03932 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 14:34:08.567942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: warnings.warn(msg) Sep 05 14:34:08.567942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: Sep 05 14:34:08.569114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4252fb-51e2-4094-9103-69ae03b03932 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.655212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:08.665920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 776/1579] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:08 2022] POST /identity/v3/projects => generated 391 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:08.665920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c78b6c-1487-47b1-9802-c7045acee021 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:08.727775 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 804/1580] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:08 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:08.727982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c78b6c-1487-47b1-9802-c7045acee021 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.729051 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4252fb-51e2-4094-9103-69ae03b03932 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:08.781919 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559f223-af7b-4eaa-b786-d808c4c125d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.783090 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559f223-af7b-4eaa-b786-d808c4c125d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.784283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559f223-af7b-4eaa-b786-d808c4c125d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.785612 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559f223-af7b-4eaa-b786-d808c4c125d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.786864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559f223-af7b-4eaa-b786-d808c4c125d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.788946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559f223-af7b-4eaa-b786-d808c4c125d8 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.789955 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4252fb-51e2-4094-9103-69ae03b03932 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.795395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2ad9a-e6af-422b-8283-331e687ecc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.796064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2ad9a-e6af-422b-8283-331e687ecc52 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.799619 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2ad9a-e6af-422b-8283-331e687ecc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.799619 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2ad9a-e6af-422b-8283-331e687ecc52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.799619 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2ad9a-e6af-422b-8283-331e687ecc52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.799619 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2ad9a-e6af-422b-8283-331e687ecc52 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.881012 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 777/1581] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:08 2022] POST /identity/v3/users => generated 295 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:08.881012 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559f223-af7b-4eaa-b786-d808c4c125d8 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:08.888787 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559f223-af7b-4eaa-b786-d808c4c125d8 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.894106 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4686f45-1991-4cf6-a5e8-6d975a210d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.894699 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4686f45-1991-4cf6-a5e8-6d975a210d72 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.895011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4686f45-1991-4cf6-a5e8-6d975a210d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.895284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4686f45-1991-4cf6-a5e8-6d975a210d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.895985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4686f45-1991-4cf6-a5e8-6d975a210d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.897581 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4686f45-1991-4cf6-a5e8-6d975a210d72 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.907780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 778/1582] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:08.908494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4686f45-1991-4cf6-a5e8-6d975a210d72 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:08.921468 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4686f45-1991-4cf6-a5e8-6d975a210d72 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:08.927064 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec69fa1-4cb2-4ebd-8c1e-d3e56177728f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:08.927759 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec69fa1-4cb2-4ebd-8c1e-d3e56177728f admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:08.928118 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec69fa1-4cb2-4ebd-8c1e-d3e56177728f admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:08.928443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec69fa1-4cb2-4ebd-8c1e-d3e56177728f admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:08.929146 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec69fa1-4cb2-4ebd-8c1e-d3e56177728f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:08.931160 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec69fa1-4cb2-4ebd-8c1e-d3e56177728f admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:08.934212 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 805/1583] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:08 2022] POST /identity/v3/projects => generated 381 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:08.935203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2ad9a-e6af-422b-8283-331e687ecc52 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.000863 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:09.004634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2ad9a-e6af-422b-8283-331e687ecc52 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.009326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954ce83-0086-4bd5-aebb-660718523370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.009988 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954ce83-0086-4bd5-aebb-660718523370 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.010271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954ce83-0086-4bd5-aebb-660718523370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.010547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954ce83-0086-4bd5-aebb-660718523370 admin admin] PATH_INFO: `/v3/projects/108a6003407b4b03ab30834f2af7d66c/users/c4f885036f6b4f9c8748666d9ecf6219/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.022332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 779/1584] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:08 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:09.025558 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec69fa1-4cb2-4ebd-8c1e-d3e56177728f admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.025695 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954ce83-0086-4bd5-aebb-660718523370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=108a6003407b4b03ab30834f2af7d66c, user_id=c4f885036f6b4f9c8748666d9ecf6219, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.028019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954ce83-0086-4bd5-aebb-660718523370 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.087019 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec69fa1-4cb2-4ebd-8c1e-d3e56177728f admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.096742 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48790024-3721-49fe-884b-4184de67e4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.097397 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48790024-3721-49fe-884b-4184de67e4e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.097684 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48790024-3721-49fe-884b-4184de67e4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.097989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48790024-3721-49fe-884b-4184de67e4e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.098650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48790024-3721-49fe-884b-4184de67e4e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.100697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48790024-3721-49fe-884b-4184de67e4e3 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.109979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:09.135194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 806/1585] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:08 2022] PUT /identity/v3/projects/108a6003407b4b03ab30834f2af7d66c/users/c4f885036f6b4f9c8748666d9ecf6219/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:09.146438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954ce83-0086-4bd5-aebb-660718523370 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.210064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954ce83-0086-4bd5-aebb-660718523370 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.215831 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ba305-4d8b-4ce5-a899-f3ba27527c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.216444 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ba305-4d8b-4ce5-a899-f3ba27527c8d admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.216727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ba305-4d8b-4ce5-a899-f3ba27527c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.217000 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ba305-4d8b-4ce5-a899-f3ba27527c8d admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.217626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ba305-4d8b-4ce5-a899-f3ba27527c8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.219811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ba305-4d8b-4ce5-a899-f3ba27527c8d admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.222050 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 780/1586] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:09 2022] POST /identity/v3/users => generated 296 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:09.226325 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48790024-3721-49fe-884b-4184de67e4e3 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.237428 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48790024-3721-49fe-884b-4184de67e4e3 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.242735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d15ee1-ae3a-4497-b651-dd347f0e25a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.243853 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d15ee1-ae3a-4497-b651-dd347f0e25a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.244253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d15ee1-ae3a-4497-b651-dd347f0e25a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.244594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d15ee1-ae3a-4497-b651-dd347f0e25a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.245305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d15ee1-ae3a-4497-b651-dd347f0e25a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.247004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d15ee1-ae3a-4497-b651-dd347f0e25a0 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.257156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 781/1587] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:09.257920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d15ee1-ae3a-4497-b651-dd347f0e25a0 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.267484 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d15ee1-ae3a-4497-b651-dd347f0e25a0 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.272373 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99301f84-00d4-49f1-8995-e928be34defb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.273720 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99301f84-00d4-49f1-8995-e928be34defb admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.274004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99301f84-00d4-49f1-8995-e928be34defb admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.274280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99301f84-00d4-49f1-8995-e928be34defb admin admin] PATH_INFO: `/v3/projects/8b61ff0e1edb4ab1becbde9e27779cda/users/fb92b28624d343fcb0abf968108baa44/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.289689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99301f84-00d4-49f1-8995-e928be34defb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b61ff0e1edb4ab1becbde9e27779cda, user_id=fb92b28624d343fcb0abf968108baa44, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.292384 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99301f84-00d4-49f1-8995-e928be34defb admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.309809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 807/1588] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:09 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:09.309809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ba305-4d8b-4ce5-a899-f3ba27527c8d admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.317060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ba305-4d8b-4ce5-a899-f3ba27527c8d admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.321977 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fca994-8323-40b1-9367-e226f5049b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.322581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fca994-8323-40b1-9367-e226f5049b9b admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.322903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fca994-8323-40b1-9367-e226f5049b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.323181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fca994-8323-40b1-9367-e226f5049b9b admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.323843 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fca994-8323-40b1-9367-e226f5049b9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.325813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fca994-8323-40b1-9367-e226f5049b9b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.386110 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 782/1589] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:09 2022] PUT /identity/v3/projects/8b61ff0e1edb4ab1becbde9e27779cda/users/fb92b28624d343fcb0abf968108baa44/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:09.386954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99301f84-00d4-49f1-8995-e928be34defb admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.418541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 808/1590] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:09 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:09.418541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6364f9-fac1-4d88-af88-b9135c28d88c None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.418541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6364f9-fac1-4d88-af88-b9135c28d88c None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.418541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6364f9-fac1-4d88-af88-b9135c28d88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.445680 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-5d6364f9-fac1-4d88-af88-b9135c28d88c None None] MFA Rules not processed for user `c4f885036f6b4f9c8748666d9ecf6219`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:09.446979 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5d6364f9-fac1-4d88-af88-b9135c28d88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:09.453554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99301f84-00d4-49f1-8995-e928be34defb admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.463404 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b27805-20ee-40e2-b0d1-4a7af2c804da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.463404 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b27805-20ee-40e2-b0d1-4a7af2c804da admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.463404 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b27805-20ee-40e2-b0d1-4a7af2c804da admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.463404 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b27805-20ee-40e2-b0d1-4a7af2c804da admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.463826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b27805-20ee-40e2-b0d1-4a7af2c804da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.465062 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b27805-20ee-40e2-b0d1-4a7af2c804da admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.515029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 809/1591] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:09.515944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6364f9-fac1-4d88-af88-b9135c28d88c None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.540872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6364f9-fac1-4d88-af88-b9135c28d88c None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.542041 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f77896-b683-4093-9a07-d685a559d437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.543366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f77896-b683-4093-9a07-d685a559d437 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.543366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f77896-b683-4093-9a07-d685a559d437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.543573 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f77896-b683-4093-9a07-d685a559d437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.544211 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f77896-b683-4093-9a07-d685a559d437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.545983 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f77896-b683-4093-9a07-d685a559d437 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.564451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 810/1592] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:09.565377 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f77896-b683-4093-9a07-d685a559d437 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.567341 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 783/1593] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:09 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:09.568595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b27805-20ee-40e2-b0d1-4a7af2c804da admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.575341 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f77896-b683-4093-9a07-d685a559d437 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.578683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b27805-20ee-40e2-b0d1-4a7af2c804da admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.580013 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2808f-1503-4249-ad72-944755d9b8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.580641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2808f-1503-4249-ad72-944755d9b8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.580930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2808f-1503-4249-ad72-944755d9b8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.581196 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2808f-1503-4249-ad72-944755d9b8d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.581830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c2808f-1503-4249-ad72-944755d9b8d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.583316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05004a-54f7-42d0-aa08-7f880b98e4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.583576 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c2808f-1503-4249-ad72-944755d9b8d0 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.583908 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05004a-54f7-42d0-aa08-7f880b98e4ec admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.584196 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05004a-54f7-42d0-aa08-7f880b98e4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.584519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05004a-54f7-42d0-aa08-7f880b98e4ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.585161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05004a-54f7-42d0-aa08-7f880b98e4ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.586750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05004a-54f7-42d0-aa08-7f880b98e4ec admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.593969 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 811/1594] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:09.595057 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd51f84-3afa-486d-b333-ef5471606496 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.595340 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd51f84-3afa-486d-b333-ef5471606496 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.595613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd51f84-3afa-486d-b333-ef5471606496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.599612 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 784/1595] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:09.600472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05004a-54f7-42d0-aa08-7f880b98e4ec admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.610534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05004a-54f7-42d0-aa08-7f880b98e4ec admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.615539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f3b28-ae2a-451e-893d-7f0c52f6b9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.616128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f3b28-ae2a-451e-893d-7f0c52f6b9fd admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.616413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f3b28-ae2a-451e-893d-7f0c52f6b9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.616690 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f3b28-ae2a-451e-893d-7f0c52f6b9fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.617322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f3b28-ae2a-451e-893d-7f0c52f6b9fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.618946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f3b28-ae2a-451e-893d-7f0c52f6b9fd admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.623793 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-edd51f84-3afa-486d-b333-ef5471606496 None None] MFA Rules not processed for user `fb92b28624d343fcb0abf968108baa44`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:09.624995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-edd51f84-3afa-486d-b333-ef5471606496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:09.630552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 785/1596] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:09.631797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c193-eb54-4901-9c57-ebef9ac4d3e2 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.639675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c193-eb54-4901-9c57-ebef9ac4d3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.639675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c193-eb54-4901-9c57-ebef9ac4d3e2 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.639675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 786/1597] 10.222.0.15 () {54 vars in 962 bytes} [Mon Sep 5 14:34:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:34:09.639675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26c193-eb54-4901-9c57-ebef9ac4d3e2 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.640135 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:09.646233 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26c193-eb54-4901-9c57-ebef9ac4d3e2 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.650764 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804804e-1973-49f6-b309-611fa2ab5968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.651357 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804804e-1973-49f6-b309-611fa2ab5968 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.651637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804804e-1973-49f6-b309-611fa2ab5968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.651939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804804e-1973-49f6-b309-611fa2ab5968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.652567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804804e-1973-49f6-b309-611fa2ab5968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.656757 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804804e-1973-49f6-b309-611fa2ab5968 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.667979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 787/1598] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:09.668804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804804e-1973-49f6-b309-611fa2ab5968 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.680788 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804804e-1973-49f6-b309-611fa2ab5968 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.685309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2faedc-358b-4288-abd3-a1018d67ab3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.685896 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2faedc-358b-4288-abd3-a1018d67ab3b admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.686176 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2faedc-358b-4288-abd3-a1018d67ab3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.686446 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2faedc-358b-4288-abd3-a1018d67ab3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.687108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2faedc-358b-4288-abd3-a1018d67ab3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.692284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2faedc-358b-4288-abd3-a1018d67ab3b admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.698517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 788/1599] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:09.698839 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 812/1600] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:09 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:09.702738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2faedc-358b-4288-abd3-a1018d67ab3b admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.702892 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd51f84-3afa-486d-b333-ef5471606496 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.711936 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd51f84-3afa-486d-b333-ef5471606496 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.713324 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2faedc-358b-4288-abd3-a1018d67ab3b admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.720910 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657fecd-6911-4acf-ba83-256abb38a878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.720910 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657fecd-6911-4acf-ba83-256abb38a878 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.721199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d02ac0-5f0f-4138-9bb9-8ee7c5cfbdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.721329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657fecd-6911-4acf-ba83-256abb38a878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.721329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657fecd-6911-4acf-ba83-256abb38a878 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.721484 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d02ac0-5f0f-4138-9bb9-8ee7c5cfbdd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.721484 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d02ac0-5f0f-4138-9bb9-8ee7c5cfbdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.721628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657fecd-6911-4acf-ba83-256abb38a878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.721702 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d02ac0-5f0f-4138-9bb9-8ee7c5cfbdd6 admin admin] PATH_INFO: `/v3/projects/834730ab3cfd4d649fc270843d47b291/users/91c691b6a7724d708bfc7424908516c8/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.721786 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657fecd-6911-4acf-ba83-256abb38a878 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.731084 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 813/1601] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:09.731925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657fecd-6911-4acf-ba83-256abb38a878 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.734862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d02ac0-5f0f-4138-9bb9-8ee7c5cfbdd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834730ab3cfd4d649fc270843d47b291, user_id=91c691b6a7724d708bfc7424908516c8, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.738071 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d02ac0-5f0f-4138-9bb9-8ee7c5cfbdd6 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.741952 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657fecd-6911-4acf-ba83-256abb38a878 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.746751 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1eca8-274d-4c83-a95b-6c1c91df082e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.749657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1eca8-274d-4c83-a95b-6c1c91df082e admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.749657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1eca8-274d-4c83-a95b-6c1c91df082e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.749657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1eca8-274d-4c83-a95b-6c1c91df082e admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.749657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1eca8-274d-4c83-a95b-6c1c91df082e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.750019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1eca8-274d-4c83-a95b-6c1c91df082e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.759715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 814/1602] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:09.760820 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf5fad-e0e9-4111-bf96-d22fdfb6b0b7 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.761123 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf5fad-e0e9-4111-bf96-d22fdfb6b0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.761373 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf5fad-e0e9-4111-bf96-d22fdfb6b0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.801947 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-15bf5fad-e0e9-4111-bf96-d22fdfb6b0b7 None None] MFA Rules not processed for user `709b9ae82dcd47408fc9a6d54a54428a`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:09.803446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-15bf5fad-e0e9-4111-bf96-d22fdfb6b0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:09.833117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 789/1603] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:09 2022] PUT /identity/v3/projects/834730ab3cfd4d649fc270843d47b291/users/91c691b6a7724d708bfc7424908516c8/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:09.834358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d02ac0-5f0f-4138-9bb9-8ee7c5cfbdd6 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.896059 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:09.899107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 815/1604] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Sep 5 14:34:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:09.899974 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf5fad-e0e9-4111-bf96-d22fdfb6b0b7 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.910580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf5fad-e0e9-4111-bf96-d22fdfb6b0b7 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.915480 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d250710e-6d34-4c7a-9371-6c7744652acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.916113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250710e-6d34-4c7a-9371-6c7744652acb admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.916400 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250710e-6d34-4c7a-9371-6c7744652acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.916674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250710e-6d34-4c7a-9371-6c7744652acb admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.917321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d250710e-6d34-4c7a-9371-6c7744652acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.919083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d250710e-6d34-4c7a-9371-6c7744652acb admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.919697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d02ac0-5f0f-4138-9bb9-8ee7c5cfbdd6 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.925394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c2a06-7bed-40c1-a6f1-b141445cd458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.926074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c2a06-7bed-40c1-a6f1-b141445cd458 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.926551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c2a06-7bed-40c1-a6f1-b141445cd458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.926865 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c2a06-7bed-40c1-a6f1-b141445cd458 admin admin] PATH_INFO: `/v3/projects/2af009055e3b4bdda73a770242253b0d/users/9136f02e6c664f27b3f4fd35475f0eea/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.929960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 816/1605] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:09.931054 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d250710e-6d34-4c7a-9371-6c7744652acb admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:09.942233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d250710e-6d34-4c7a-9371-6c7744652acb admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:09.945273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c2a06-7bed-40c1-a6f1-b141445cd458 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af009055e3b4bdda73a770242253b0d, user_id=9136f02e6c664f27b3f4fd35475f0eea, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.948097 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c2a06-7bed-40c1-a6f1-b141445cd458 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:09.948319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ff415-3935-4674-b2e8-332c849415db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:09.949088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ff415-3935-4674-b2e8-332c849415db admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:09.949519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ff415-3935-4674-b2e8-332c849415db admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:09.949942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ff415-3935-4674-b2e8-332c849415db admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:09.950812 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ff415-3935-4674-b2e8-332c849415db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:09.952873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ff415-3935-4674-b2e8-332c849415db admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:10.043373 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 790/1606] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:09 2022] PUT /identity/v3/projects/2af009055e3b4bdda73a770242253b0d/users/9136f02e6c664f27b3f4fd35475f0eea/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:10.044176 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c2a06-7bed-40c1-a6f1-b141445cd458 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:10.058841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 817/1607] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:09 2022] POST /identity/v3/projects => generated 405 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:10.059562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ff415-3935-4674-b2e8-332c849415db admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:10.106310 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:10.127975 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c2a06-7bed-40c1-a6f1-b141445cd458 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:10.133831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bf3f6-992f-4009-96d3-c0ac139d942c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:10.135248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bf3f6-992f-4009-96d3-c0ac139d942c admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:10.135534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bf3f6-992f-4009-96d3-c0ac139d942c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:10.135809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bf3f6-992f-4009-96d3-c0ac139d942c admin admin] PATH_INFO: `/v3/projects/11dfac0c6f8f47b9a02dbcc53db203e4/users/bbb6b8c6d8d04e72a1fc06dfa2bb3692/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:10.140129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ff415-3935-4674-b2e8-332c849415db admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:10.146234 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95f63b-c4e2-41c1-ad32-5892fd07c41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:10.147197 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95f63b-c4e2-41c1-ad32-5892fd07c41a admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:10.147197 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95f63b-c4e2-41c1-ad32-5892fd07c41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:10.147532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95f63b-c4e2-41c1-ad32-5892fd07c41a admin admin] PATH_INFO: `/v3/projects/31e97d66fc9d4f51ac0849d5ee258907/users/87e458756a534cb38c823e884b3d7569/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:10.155973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bf3f6-992f-4009-96d3-c0ac139d942c admin admin] RBAC: Authorizing `identity:create_grant(project_id=11dfac0c6f8f47b9a02dbcc53db203e4, user_id=bbb6b8c6d8d04e72a1fc06dfa2bb3692, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:10.159901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bf3f6-992f-4009-96d3-c0ac139d942c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:10.163610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95f63b-c4e2-41c1-ad32-5892fd07c41a admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e97d66fc9d4f51ac0849d5ee258907, user_id=87e458756a534cb38c823e884b3d7569, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:10.166778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95f63b-c4e2-41c1-ad32-5892fd07c41a admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:10.283576 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 818/1608] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:10 2022] PUT /identity/v3/projects/31e97d66fc9d4f51ac0849d5ee258907/users/87e458756a534cb38c823e884b3d7569/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:10.284380 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 791/1609] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:10 2022] PUT /identity/v3/projects/11dfac0c6f8f47b9a02dbcc53db203e4/users/bbb6b8c6d8d04e72a1fc06dfa2bb3692/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:10.285060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bf3f6-992f-4009-96d3-c0ac139d942c admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:10.295469 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0da8b0-ba42-4561-920d-25dc811f2eb3 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:10.295781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0da8b0-ba42-4561-920d-25dc811f2eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:10.296068 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0da8b0-ba42-4561-920d-25dc811f2eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:10.314531 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-6d0da8b0-ba42-4561-920d-25dc811f2eb3 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:10.316522 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-6d0da8b0-ba42-4561-920d-25dc811f2eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:10.392243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bf3f6-992f-4009-96d3-c0ac139d942c admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:10.398518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 819/1610] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:10.398518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0da8b0-ba42-4561-920d-25dc811f2eb3 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:10.403047 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bf821-87d9-4359-be0f-fe87e8a6ca78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:10.403873 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bf821-87d9-4359-be0f-fe87e8a6ca78 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:10.404165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bf821-87d9-4359-be0f-fe87e8a6ca78 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:10.404442 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bf821-87d9-4359-be0f-fe87e8a6ca78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:10.410924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0da8b0-ba42-4561-920d-25dc811f2eb3 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:10.417457 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa77650-84a4-4475-99fc-897266072acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:10.418640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa77650-84a4-4475-99fc-897266072acb admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:10.418961 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa77650-84a4-4475-99fc-897266072acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:10.419397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa77650-84a4-4475-99fc-897266072acb admin admin] PATH_INFO: `/v3/projects/3c30f843007d4200ac64e2189003698f/users/d31427f2a67e49d0b4d3e1b3cee8c48b/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:10.442858 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa77650-84a4-4475-99fc-897266072acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c30f843007d4200ac64e2189003698f, user_id=d31427f2a67e49d0b4d3e1b3cee8c48b, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:10.445729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa77650-84a4-4475-99fc-897266072acb admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:10.454412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:10.471665 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32bf821-87d9-4359-be0f-fe87e8a6ca78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:10.473520 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32bf821-87d9-4359-be0f-fe87e8a6ca78 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:10.483216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 792/1611] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:10.484648 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f453547-52ea-4fae-972e-566e0471c40f None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:10.484910 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f453547-52ea-4fae-972e-566e0471c40f None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:10.485178 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f453547-52ea-4fae-972e-566e0471c40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:10.505516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-4f453547-52ea-4fae-972e-566e0471c40f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:10.506727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-4f453547-52ea-4fae-972e-566e0471c40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:10.517835 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 793/1612] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:10.519349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f453547-52ea-4fae-972e-566e0471c40f None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:10.529824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 820/1613] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:10 2022] PUT /identity/v3/projects/3c30f843007d4200ac64e2189003698f/users/d31427f2a67e49d0b4d3e1b3cee8c48b/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:10.529824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a044a-cebe-4825-9bb2-d43069c29a2c None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:10.529824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a044a-cebe-4825-9bb2-d43069c29a2c None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:10.529824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a044a-cebe-4825-9bb2-d43069c29a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:10.542618 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f453547-52ea-4fae-972e-566e0471c40f None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:10.588341 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-509a044a-cebe-4825-9bb2-d43069c29a2c None None] MFA Rules not processed for user `87e458756a534cb38c823e884b3d7569`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:10.590129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-509a044a-cebe-4825-9bb2-d43069c29a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:10.645134 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535bf77-3341-49a0-a10b-05aa6687ac10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:10.646053 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535bf77-3341-49a0-a10b-05aa6687ac10 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:10.646325 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535bf77-3341-49a0-a10b-05aa6687ac10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:10.646728 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535bf77-3341-49a0-a10b-05aa6687ac10 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:10.647793 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535bf77-3341-49a0-a10b-05aa6687ac10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:10.650446 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535bf77-3341-49a0-a10b-05aa6687ac10 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:10.693068 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 821/1614] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:10.694666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50e8af-a6c5-41d4-b50c-93514d0d25d9 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:10.695094 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50e8af-a6c5-41d4-b50c-93514d0d25d9 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:10.695693 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50e8af-a6c5-41d4-b50c-93514d0d25d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:10.731149 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-1c50e8af-a6c5-41d4-b50c-93514d0d25d9 None None] MFA Rules not processed for user `bbb6b8c6d8d04e72a1fc06dfa2bb3692`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:10.732430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-1c50e8af-a6c5-41d4-b50c-93514d0d25d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:10.753385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 794/1615] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:10 2022] POST /identity/v3/users => generated 309 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:10.754695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535bf77-3341-49a0-a10b-05aa6687ac10 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:10.761177 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:10.827424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535bf77-3341-49a0-a10b-05aa6687ac10 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:10.829188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 822/1616] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:10.831557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9d5fd-761b-477c-816c-2df37406eb99 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:10.831845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9d5fd-761b-477c-816c-2df37406eb99 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:10.832118 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9d5fd-761b-477c-816c-2df37406eb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:10.850707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d07acd-4926-4190-93d0-a3262138146a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:10.851480 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d07acd-4926-4190-93d0-a3262138146a service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:10.851866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d07acd-4926-4190-93d0-a3262138146a service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:10.852244 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d07acd-4926-4190-93d0-a3262138146a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:10.867869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d07acd-4926-4190-93d0-a3262138146a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:10.869610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d07acd-4926-4190-93d0-a3262138146a service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:10.882292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-48a9d5fd-761b-477c-816c-2df37406eb99 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:10.882292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-48a9d5fd-761b-477c-816c-2df37406eb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:10.883854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 795/1617] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:10.884721 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d07acd-4926-4190-93d0-a3262138146a service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:10.897369 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d07acd-4926-4190-93d0-a3262138146a service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:10.899026 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 823/1618] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:10.900356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9d5fd-761b-477c-816c-2df37406eb99 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:10.902211 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf14a3-fb09-47d2-bb54-391f57d0f1e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:10.903452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf14a3-fb09-47d2-bb54-391f57d0f1e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:10.905924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf14a3-fb09-47d2-bb54-391f57d0f1e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:10.905924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf14a3-fb09-47d2-bb54-391f57d0f1e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:10.916548 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9d5fd-761b-477c-816c-2df37406eb99 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:10.917651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf14a3-fb09-47d2-bb54-391f57d0f1e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:10.919189 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf14a3-fb09-47d2-bb54-391f57d0f1e8 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:10.921594 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cba829-64c3-43ff-b3c9-abf9d416ea58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:10.922400 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cba829-64c3-43ff-b3c9-abf9d416ea58 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:10.922880 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cba829-64c3-43ff-b3c9-abf9d416ea58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:10.923304 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cba829-64c3-43ff-b3c9-abf9d416ea58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:10.924123 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cba829-64c3-43ff-b3c9-abf9d416ea58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:10.926595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cba829-64c3-43ff-b3c9-abf9d416ea58 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:10.936567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 796/1619] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:10.937816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccf3eb-5f30-4582-a425-59939ec128bb None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:10.938107 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccf3eb-5f30-4582-a425-59939ec128bb None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:10.939286 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccf3eb-5f30-4582-a425-59939ec128bb None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:10.945419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 797/1620] 10.222.0.15 () {54 vars in 912 bytes} [Mon Sep 5 14:34:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:34:10.946386 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbccf3eb-5f30-4582-a425-59939ec128bb None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:10.962110 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbccf3eb-5f30-4582-a425-59939ec128bb None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:10.967615 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60ed82-893b-4978-af65-779ad1349d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:10.968334 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60ed82-893b-4978-af65-779ad1349d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:10.968686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60ed82-893b-4978-af65-779ad1349d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:10.969029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60ed82-893b-4978-af65-779ad1349d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:10.969673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60ed82-893b-4978-af65-779ad1349d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:10.971537 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60ed82-893b-4978-af65-779ad1349d2c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:10.987594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 798/1621] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:10.987594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00f05b-2acd-43a6-983c-6e030ddfbcf7 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:10.987594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00f05b-2acd-43a6-983c-6e030ddfbcf7 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:10.987594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00f05b-2acd-43a6-983c-6e030ddfbcf7 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:10.987594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 799/1622] 10.222.0.15 () {54 vars in 962 bytes} [Mon Sep 5 14:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:34:10.987594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00f05b-2acd-43a6-983c-6e030ddfbcf7 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:10.998035 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00f05b-2acd-43a6-983c-6e030ddfbcf7 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:11.001863 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da729b-0035-45bc-9ec8-1f8a125963a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:11.003034 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da729b-0035-45bc-9ec8-1f8a125963a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:11.003313 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da729b-0035-45bc-9ec8-1f8a125963a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:11.003641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da729b-0035-45bc-9ec8-1f8a125963a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:11.013509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da729b-0035-45bc-9ec8-1f8a125963a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:11.015762 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da729b-0035-45bc-9ec8-1f8a125963a6 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:11.025905 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 800/1623] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:11.027512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b994b-a31e-4e1b-b58e-3f32cd2afd5a None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:11.027921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b994b-a31e-4e1b-b58e-3f32cd2afd5a None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:11.028370 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b994b-a31e-4e1b-b58e-3f32cd2afd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:11.055015 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-6c4b994b-a31e-4e1b-b58e-3f32cd2afd5a None None] MFA Rules not processed for user `709b9ae82dcd47408fc9a6d54a54428a`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:11.056343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-6c4b994b-a31e-4e1b-b58e-3f32cd2afd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:11.062266 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 824/1624] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:10 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:11.063336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cba829-64c3-43ff-b3c9-abf9d416ea58 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:11.093236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:11.280289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cba829-64c3-43ff-b3c9-abf9d416ea58 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:11.281186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 801/1625] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 5 14:34:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:11.281186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b5104-cff2-4fd8-8762-7773c57e23dd None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:11.281186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b5104-cff2-4fd8-8762-7773c57e23dd None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:11.281186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b5104-cff2-4fd8-8762-7773c57e23dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:11.281701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f72648-1b50-44e6-aa3c-9f47c72a5c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:11.281701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f72648-1b50-44e6-aa3c-9f47c72a5c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:11.281701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f72648-1b50-44e6-aa3c-9f47c72a5c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:11.281701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f72648-1b50-44e6-aa3c-9f47c72a5c54 admin admin] PATH_INFO: `/v3/projects/a365645db29f4ea4a1bc943669ae9eff/users/94bb1ffa99ce4da8beab71505cc7716d/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:11.282117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f72648-1b50-44e6-aa3c-9f47c72a5c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a365645db29f4ea4a1bc943669ae9eff, user_id=94bb1ffa99ce4da8beab71505cc7716d, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:11.282117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f72648-1b50-44e6-aa3c-9f47c72a5c54 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:11.282330 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-1b4b5104-cff2-4fd8-8762-7773c57e23dd None None] MFA Rules not processed for user `709b9ae82dcd47408fc9a6d54a54428a`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:11.282330 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-1b4b5104-cff2-4fd8-8762-7773c57e23dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:11.282330 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 802/1626] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Sep 5 14:34:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:11.282330 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b5104-cff2-4fd8-8762-7773c57e23dd None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:11.292257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b5104-cff2-4fd8-8762-7773c57e23dd None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:11.303251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e4dcb-b240-443a-a92f-5faf7b3c9d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:11.303538 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e4dcb-b240-443a-a92f-5faf7b3c9d5f admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:11.304789 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e4dcb-b240-443a-a92f-5faf7b3c9d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:11.304789 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e4dcb-b240-443a-a92f-5faf7b3c9d5f admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:11.305185 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e4dcb-b240-443a-a92f-5faf7b3c9d5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:11.308065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e4dcb-b240-443a-a92f-5faf7b3c9d5f admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:11.357307 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 825/1627] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:11 2022] PUT /identity/v3/projects/a365645db29f4ea4a1bc943669ae9eff/users/94bb1ffa99ce4da8beab71505cc7716d/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:11.360317 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f72648-1b50-44e6-aa3c-9f47c72a5c54 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:11.437301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 803/1628] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:11 2022] POST /identity/v3/users => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:11.442050 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e4dcb-b240-443a-a92f-5faf7b3c9d5f admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:11.453796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f72648-1b50-44e6-aa3c-9f47c72a5c54 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:11.455242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e4dcb-b240-443a-a92f-5faf7b3c9d5f admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:11.458498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49c118-d415-410e-b612-1b7419ab0f7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:11.460321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49c118-d415-410e-b612-1b7419ab0f7d service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:11.460696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49c118-d415-410e-b612-1b7419ab0f7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:11.461060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49c118-d415-410e-b612-1b7419ab0f7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:11.485379 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:11.536137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fab75-1caa-4ba0-86aa-d356f61b84f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:11.537088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fab75-1caa-4ba0-86aa-d356f61b84f7 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:11.537481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fab75-1caa-4ba0-86aa-d356f61b84f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:11.537827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fab75-1caa-4ba0-86aa-d356f61b84f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:11.556332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49c118-d415-410e-b612-1b7419ab0f7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:11.561633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49c118-d415-410e-b612-1b7419ab0f7d service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:11.576322 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 826/1629] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:11.577284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49c118-d415-410e-b612-1b7419ab0f7d service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:11.638989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fab75-1caa-4ba0-86aa-d356f61b84f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:11.640679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fab75-1caa-4ba0-86aa-d356f61b84f7 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:11.653452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 804/1630] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:11.654605 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fab75-1caa-4ba0-86aa-d356f61b84f7 service nova] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:11.681897 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49c118-d415-410e-b612-1b7419ab0f7d service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:11.689090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15939b0d-42dc-4d75-bec8-cbe6e48b4f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:11.690899 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15939b0d-42dc-4d75-bec8-cbe6e48b4f78 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:11.691249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15939b0d-42dc-4d75-bec8-cbe6e48b4f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:11.691533 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15939b0d-42dc-4d75-bec8-cbe6e48b4f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:11.692194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15939b0d-42dc-4d75-bec8-cbe6e48b4f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:11.694634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15939b0d-42dc-4d75-bec8-cbe6e48b4f78 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:11.714402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 827/1631] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:11.719941 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15939b0d-42dc-4d75-bec8-cbe6e48b4f78 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:11.742754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15939b0d-42dc-4d75-bec8-cbe6e48b4f78 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:11.755043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6821e36-f985-4bef-883c-9956da6b5d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:11.755775 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6821e36-f985-4bef-883c-9956da6b5d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:11.756155 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6821e36-f985-4bef-883c-9956da6b5d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:11.756342 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6821e36-f985-4bef-883c-9956da6b5d59 admin admin] PATH_INFO: `/v3/projects/a365645db29f4ea4a1bc943669ae9eff/users/94bb1ffa99ce4da8beab71505cc7716d/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:11.758429 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6821e36-f985-4bef-883c-9956da6b5d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a365645db29f4ea4a1bc943669ae9eff, user_id=94bb1ffa99ce4da8beab71505cc7716d, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:11.762045 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6821e36-f985-4bef-883c-9956da6b5d59 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:11.783151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fab75-1caa-4ba0-86aa-d356f61b84f7 service nova] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:11.784434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 828/1632] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:11 2022] PUT /identity/v3/projects/a365645db29f4ea4a1bc943669ae9eff/users/94bb1ffa99ce4da8beab71505cc7716d/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:11.802059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb0866-fae1-4cbd-bf80-2e96d5e90536 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:11.802592 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb0866-fae1-4cbd-bf80-2e96d5e90536 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:11.803489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb0866-fae1-4cbd-bf80-2e96d5e90536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:11.886050 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-b2cb0866-fae1-4cbd-bf80-2e96d5e90536 None None] MFA Rules not processed for user `94bb1ffa99ce4da8beab71505cc7716d`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:11.887654 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b2cb0866-fae1-4cbd-bf80-2e96d5e90536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:11.951987 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:11.970096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93c823-b6de-472f-bf93-119ab29527b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:11.970780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93c823-b6de-472f-bf93-119ab29527b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:11.971515 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93c823-b6de-472f-bf93-119ab29527b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:11.971804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93c823-b6de-472f-bf93-119ab29527b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:11.974823 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93c823-b6de-472f-bf93-119ab29527b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:11.977719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93c823-b6de-472f-bf93-119ab29527b3 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:12.035023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 805/1633] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:12.044725 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93c823-b6de-472f-bf93-119ab29527b3 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:12.058193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 829/1634] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:11 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:34:12.072853 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb0866-fae1-4cbd-bf80-2e96d5e90536 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:12.076693 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93c823-b6de-472f-bf93-119ab29527b3 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:12.085203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0ed9f-520c-4d9e-a82a-eebc0f1444bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:12.090077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0ed9f-520c-4d9e-a82a-eebc0f1444bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:12.090462 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0ed9f-520c-4d9e-a82a-eebc0f1444bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:12.090847 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0ed9f-520c-4d9e-a82a-eebc0f1444bd admin admin] PATH_INFO: `/v3/projects/33a94d38f1be4967b127233847b0fc28/users/b8ecac515c894105818cde6bfcc32542/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:12.138713 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0ed9f-520c-4d9e-a82a-eebc0f1444bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=33a94d38f1be4967b127233847b0fc28, user_id=b8ecac515c894105818cde6bfcc32542, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:12.146547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0ed9f-520c-4d9e-a82a-eebc0f1444bd admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:12.195653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb0866-fae1-4cbd-bf80-2e96d5e90536 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:12.199008 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10fe56-2652-4423-933a-4ac1ca8fbf74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:12.199636 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10fe56-2652-4423-933a-4ac1ca8fbf74 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:12.199922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10fe56-2652-4423-933a-4ac1ca8fbf74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:12.200198 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10fe56-2652-4423-933a-4ac1ca8fbf74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:12.211072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10fe56-2652-4423-933a-4ac1ca8fbf74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:12.213245 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10fe56-2652-4423-933a-4ac1ca8fbf74 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:12.229569 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 830/1635] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:12 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:12.229569 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b91d47-0821-4b84-82d9-2d8a1a917cb2 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:12.229569 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b91d47-0821-4b84-82d9-2d8a1a917cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:12.229569 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b91d47-0821-4b84-82d9-2d8a1a917cb2 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:12.230446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 831/1636] 10.222.0.15 () {54 vars in 912 bytes} [Mon Sep 5 14:34:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:34:12.236437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09d8a0-b2e0-4e73-b847-33ef573391d3 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:12.236871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09d8a0-b2e0-4e73-b847-33ef573391d3 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:12.237330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09d8a0-b2e0-4e73-b847-33ef573391d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:12.258244 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 806/1637] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:12 2022] PUT /identity/v3/projects/33a94d38f1be4967b127233847b0fc28/users/b8ecac515c894105818cde6bfcc32542/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:12.258894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:12.261936 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0ed9f-520c-4d9e-a82a-eebc0f1444bd admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:12.308797 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-9a09d8a0-b2e0-4e73-b847-33ef573391d3 None None] MFA Rules not processed for user `709b9ae82dcd47408fc9a6d54a54428a`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:12.310910 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-9a09d8a0-b2e0-4e73-b847-33ef573391d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:12.431051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0ed9f-520c-4d9e-a82a-eebc0f1444bd admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:12.677816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c333a681-09c3-468e-b150-55174e1d9bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:12.677816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c333a681-09c3-468e-b150-55174e1d9bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:12.677816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c333a681-09c3-468e-b150-55174e1d9bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:12.677816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c333a681-09c3-468e-b150-55174e1d9bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:12.679199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c333a681-09c3-468e-b150-55174e1d9bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:12.679199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c333a681-09c3-468e-b150-55174e1d9bd9 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:12.679199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 807/1638] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:12.679199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c333a681-09c3-468e-b150-55174e1d9bd9 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:12.679199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c333a681-09c3-468e-b150-55174e1d9bd9 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:12.681262 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988b2ab-f7ad-483e-9efa-ad7e750959a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:12.681262 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988b2ab-f7ad-483e-9efa-ad7e750959a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:12.681262 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988b2ab-f7ad-483e-9efa-ad7e750959a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:12.681262 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988b2ab-f7ad-483e-9efa-ad7e750959a7 admin admin] PATH_INFO: `/v3/projects/33a94d38f1be4967b127233847b0fc28/users/b8ecac515c894105818cde6bfcc32542/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:12.682191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988b2ab-f7ad-483e-9efa-ad7e750959a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33a94d38f1be4967b127233847b0fc28, user_id=b8ecac515c894105818cde6bfcc32542, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:12.682191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988b2ab-f7ad-483e-9efa-ad7e750959a7 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:12.682564 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 832/1639] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 5 14:34:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:34:12.682887 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 808/1640] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:12 2022] PUT /identity/v3/projects/33a94d38f1be4967b127233847b0fc28/users/b8ecac515c894105818cde6bfcc32542/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:34:12.683158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacafb7-6995-4e75-9b76-9607f909c5d4 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:12.683453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988b2ab-f7ad-483e-9efa-ad7e750959a7 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:12.683646 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacafb7-6995-4e75-9b76-9607f909c5d4 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:12.683646 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacafb7-6995-4e75-9b76-9607f909c5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:12.683646 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-6eacafb7-6995-4e75-9b76-9607f909c5d4 None None] MFA Rules not processed for user `b8ecac515c894105818cde6bfcc32542`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:12.683646 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-6eacafb7-6995-4e75-9b76-9607f909c5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:12.720395 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988b2ab-f7ad-483e-9efa-ad7e750959a7 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:12.720395 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579d297-33d0-4228-aa28-44297877e768 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:12.723189 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579d297-33d0-4228-aa28-44297877e768 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:12.723189 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579d297-33d0-4228-aa28-44297877e768 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:12.725621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579d297-33d0-4228-aa28-44297877e768 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:12.822594 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 833/1641] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:12.860967 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579d297-33d0-4228-aa28-44297877e768 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:12.863164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579d297-33d0-4228-aa28-44297877e768 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:12.876884 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 809/1642] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:13.058948 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:13.200156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacafb7-6995-4e75-9b76-9607f909c5d4 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:13.220329 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579d297-33d0-4228-aa28-44297877e768 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:13.237482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579d297-33d0-4228-aa28-44297877e768 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:13.255275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0db07-9d33-423b-8860-21a88e9522f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:13.257486 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0db07-9d33-423b-8860-21a88e9522f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:13.258480 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0db07-9d33-423b-8860-21a88e9522f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:13.260941 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0db07-9d33-423b-8860-21a88e9522f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:13.295768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a0db07-9d33-423b-8860-21a88e9522f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:13.303173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a0db07-9d33-423b-8860-21a88e9522f1 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:13.342390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 810/1643] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:34:13.440325 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:13.443022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacafb7-6995-4e75-9b76-9607f909c5d4 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:13.460611 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e006d-1d60-4b3b-884b-3c71dc508ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:13.462140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e006d-1d60-4b3b-884b-3c71dc508ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:13.463880 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e006d-1d60-4b3b-884b-3c71dc508ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:13.464161 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e006d-1d60-4b3b-884b-3c71dc508ccc admin admin] PATH_INFO: `/v3/users/fb92b28624d343fcb0abf968108baa44` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:13.468219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e006d-1d60-4b3b-884b-3c71dc508ccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb92b28624d343fcb0abf968108baa44)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:13.470831 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: WARNING py.warnings [None req-d54e006d-1d60-4b3b-884b-3c71dc508ccc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 14:34:13.470831 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: warnings.warn(msg) Sep 05 14:34:13.470831 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: Sep 05 14:34:13.476227 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e006d-1d60-4b3b-884b-3c71dc508ccc admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:13.754437 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0db07-9d33-423b-8860-21a88e9522f1 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:13.767607 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0db07-9d33-423b-8860-21a88e9522f1 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:13.779545 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5321ca-2eb5-4c44-8080-0f425de05147 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:13.780384 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5321ca-2eb5-4c44-8080-0f425de05147 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:13.780851 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5321ca-2eb5-4c44-8080-0f425de05147 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:13.781286 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5321ca-2eb5-4c44-8080-0f425de05147 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:13.879263 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5321ca-2eb5-4c44-8080-0f425de05147 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:13.897113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5321ca-2eb5-4c44-8080-0f425de05147 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:13.903301 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d54e006d-1d60-4b3b-884b-3c71dc508ccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb92b28624d343fcb0abf968108baa44'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:13.937110 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 811/1644] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:34:13.982090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d54e006d-1d60-4b3b-884b-3c71dc508ccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb92b28624d343fcb0abf968108baa44'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:14.066515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d54e006d-1d60-4b3b-884b-3c71dc508ccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb92b28624d343fcb0abf968108baa44'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:14.069576 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d54e006d-1d60-4b3b-884b-3c71dc508ccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb92b28624d343fcb0abf968108baa44'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:14.273091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d54e006d-1d60-4b3b-884b-3c71dc508ccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb92b28624d343fcb0abf968108baa44'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:14.273091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 834/1645] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:34:13 2022] DELETE /identity/v3/users/fb92b28624d343fcb0abf968108baa44 => generated 0 bytes in 932 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:14.275309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a5b9f-82c0-48f6-b3ea-b20db1d77227 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:14.275309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a5b9f-82c0-48f6-b3ea-b20db1d77227 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:14.275309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a5b9f-82c0-48f6-b3ea-b20db1d77227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:14.275309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-f76a5b9f-82c0-48f6-b3ea-b20db1d77227 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:14.275309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f76a5b9f-82c0-48f6-b3ea-b20db1d77227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:14.366829 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 812/1646] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:14.372320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e006d-1d60-4b3b-884b-3c71dc508ccc admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:14.373859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d54e006d-1d60-4b3b-884b-3c71dc508ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:14.533053 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e006d-1d60-4b3b-884b-3c71dc508ccc admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:14.544425 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafd300-3a38-4349-b226-165349429134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:14.546577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafd300-3a38-4349-b226-165349429134 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:14.554003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafd300-3a38-4349-b226-165349429134 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:14.554003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafd300-3a38-4349-b226-165349429134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:14.568858 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafd300-3a38-4349-b226-165349429134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:14.570542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafd300-3a38-4349-b226-165349429134 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:14.583219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 835/1647] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:34:16.499908 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a5b9f-82c0-48f6-b3ea-b20db1d77227 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:16.501997 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f76a5b9f-82c0-48f6-b3ea-b20db1d77227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:16.522226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:16.531965 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a5b9f-82c0-48f6-b3ea-b20db1d77227 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:16.563013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ed409-6124-4a55-b74c-c43d001fbbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:16.566251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ed409-6124-4a55-b74c-c43d001fbbc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:16.566586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ed409-6124-4a55-b74c-c43d001fbbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:16.567167 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ed409-6124-4a55-b74c-c43d001fbbc6 admin admin] PATH_INFO: `/v3/projects/8b61ff0e1edb4ab1becbde9e27779cda` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:16.583828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ed409-6124-4a55-b74c-c43d001fbbc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b61ff0e1edb4ab1becbde9e27779cda)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:16.583828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: WARNING py.warnings [None req-466ed409-6124-4a55-b74c-c43d001fbbc6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 14:34:16.583828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: warnings.warn(msg) Sep 05 14:34:16.583828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: Sep 05 14:34:16.583828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ed409-6124-4a55-b74c-c43d001fbbc6 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:16.991191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-466ed409-6124-4a55-b74c-c43d001fbbc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b61ff0e1edb4ab1becbde9e27779cda', 'request_id': 'req-466ed409-6124-4a55-b74c-c43d001fbbc6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:17.018231 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-466ed409-6124-4a55-b74c-c43d001fbbc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b61ff0e1edb4ab1becbde9e27779cda', 'request_id': 'req-466ed409-6124-4a55-b74c-c43d001fbbc6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:17.173917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-466ed409-6124-4a55-b74c-c43d001fbbc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b61ff0e1edb4ab1becbde9e27779cda', 'request_id': 'req-466ed409-6124-4a55-b74c-c43d001fbbc6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:17.217236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-466ed409-6124-4a55-b74c-c43d001fbbc6 admin admin] The token cache is being invalidate because project 8b61ff0e1edb4ab1becbde9e27779cda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:34:17.218488 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-466ed409-6124-4a55-b74c-c43d001fbbc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:17.225124 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-466ed409-6124-4a55-b74c-c43d001fbbc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:17.233008 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 813/1648] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:34:16 2022] DELETE /identity/v3/projects/8b61ff0e1edb4ab1becbde9e27779cda => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:17.234038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafd300-3a38-4349-b226-165349429134 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:17.246449 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafd300-3a38-4349-b226-165349429134 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:17.357699 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:17.381214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80b55f-1eca-4a6e-b5b2-7ada76b49b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:17.381937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80b55f-1eca-4a6e-b5b2-7ada76b49b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:17.382203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80b55f-1eca-4a6e-b5b2-7ada76b49b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:17.384554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80b55f-1eca-4a6e-b5b2-7ada76b49b01 admin admin] PATH_INFO: `/v3/users/94bb1ffa99ce4da8beab71505cc7716d` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:17.389587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80b55f-1eca-4a6e-b5b2-7ada76b49b01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94bb1ffa99ce4da8beab71505cc7716d)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:17.392343 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80b55f-1eca-4a6e-b5b2-7ada76b49b01 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:17.690471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8a80b55f-1eca-4a6e-b5b2-7ada76b49b01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94bb1ffa99ce4da8beab71505cc7716d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:17.777200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8a80b55f-1eca-4a6e-b5b2-7ada76b49b01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94bb1ffa99ce4da8beab71505cc7716d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:17.821341 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8a80b55f-1eca-4a6e-b5b2-7ada76b49b01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94bb1ffa99ce4da8beab71505cc7716d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:17.822139 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8a80b55f-1eca-4a6e-b5b2-7ada76b49b01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94bb1ffa99ce4da8beab71505cc7716d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:17.833809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8a80b55f-1eca-4a6e-b5b2-7ada76b49b01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94bb1ffa99ce4da8beab71505cc7716d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:17.845365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 836/1649] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:34:17 2022] DELETE /identity/v3/users/94bb1ffa99ce4da8beab71505cc7716d => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:34:17.998088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ed409-6124-4a55-b74c-c43d001fbbc6 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:17.998088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-466ed409-6124-4a55-b74c-c43d001fbbc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:18.014909 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ed409-6124-4a55-b74c-c43d001fbbc6 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:18.088875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f755697-d394-4193-bc8d-d85c278941c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:18.089763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f755697-d394-4193-bc8d-d85c278941c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:18.090104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f755697-d394-4193-bc8d-d85c278941c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:18.090595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f755697-d394-4193-bc8d-d85c278941c5 admin admin] PATH_INFO: `/v3/users/c4f885036f6b4f9c8748666d9ecf6219` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:18.095680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f755697-d394-4193-bc8d-d85c278941c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4f885036f6b4f9c8748666d9ecf6219)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:18.097924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f755697-d394-4193-bc8d-d85c278941c5 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:18.359723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4f755697-d394-4193-bc8d-d85c278941c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4f885036f6b4f9c8748666d9ecf6219'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:18.548126 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4f755697-d394-4193-bc8d-d85c278941c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4f885036f6b4f9c8748666d9ecf6219'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:18.548126 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4f755697-d394-4193-bc8d-d85c278941c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4f885036f6b4f9c8748666d9ecf6219'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:18.590180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4f755697-d394-4193-bc8d-d85c278941c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4f885036f6b4f9c8748666d9ecf6219'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:18.619158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4f755697-d394-4193-bc8d-d85c278941c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4f885036f6b4f9c8748666d9ecf6219'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:18.630892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 814/1650] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:34:17 2022] DELETE /identity/v3/users/c4f885036f6b4f9c8748666d9ecf6219 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:18.639362 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ac7a0-855c-48b2-b043-8b5e64fe86c0 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:18.639362 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ac7a0-855c-48b2-b043-8b5e64fe86c0 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:18.639362 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ac7a0-855c-48b2-b043-8b5e64fe86c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:18.669068 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-f99ac7a0-855c-48b2-b043-8b5e64fe86c0 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:18.672581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f99ac7a0-855c-48b2-b043-8b5e64fe86c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:18.823909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 837/1651] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:18.901647 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f755697-d394-4193-bc8d-d85c278941c5 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:18.903837 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-4f755697-d394-4193-bc8d-d85c278941c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:18.915000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:19.040061 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f755697-d394-4193-bc8d-d85c278941c5 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:19.045909 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e409a5-6e05-44d7-be5d-d5ec0de3fd3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:19.048137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e409a5-6e05-44d7-be5d-d5ec0de3fd3b service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:19.049892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e409a5-6e05-44d7-be5d-d5ec0de3fd3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:19.050723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e409a5-6e05-44d7-be5d-d5ec0de3fd3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:19.067866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e409a5-6e05-44d7-be5d-d5ec0de3fd3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:19.069428 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e409a5-6e05-44d7-be5d-d5ec0de3fd3b service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:19.087295 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 815/1652] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:19.523603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ac7a0-855c-48b2-b043-8b5e64fe86c0 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:19.526130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f99ac7a0-855c-48b2-b043-8b5e64fe86c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:19.553424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ac7a0-855c-48b2-b043-8b5e64fe86c0 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:19.571928 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208228c-65a6-4776-aead-ae4f22b48c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:19.573801 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208228c-65a6-4776-aead-ae4f22b48c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:19.574093 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208228c-65a6-4776-aead-ae4f22b48c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:19.574367 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208228c-65a6-4776-aead-ae4f22b48c60 admin admin] PATH_INFO: `/v3/projects/a365645db29f4ea4a1bc943669ae9eff` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:19.576247 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208228c-65a6-4776-aead-ae4f22b48c60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a365645db29f4ea4a1bc943669ae9eff)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:19.589512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208228c-65a6-4776-aead-ae4f22b48c60 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:19.766450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a208228c-65a6-4776-aead-ae4f22b48c60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a365645db29f4ea4a1bc943669ae9eff', 'request_id': 'req-a208228c-65a6-4776-aead-ae4f22b48c60'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:19.776352 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a208228c-65a6-4776-aead-ae4f22b48c60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a365645db29f4ea4a1bc943669ae9eff', 'request_id': 'req-a208228c-65a6-4776-aead-ae4f22b48c60'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:19.838421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41008cbc-e93e-4c8b-ad50-0a5bc794c7cb None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:19.838979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41008cbc-e93e-4c8b-ad50-0a5bc794c7cb None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:19.839470 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41008cbc-e93e-4c8b-ad50-0a5bc794c7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:19.866073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a208228c-65a6-4776-aead-ae4f22b48c60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a365645db29f4ea4a1bc943669ae9eff', 'request_id': 'req-a208228c-65a6-4776-aead-ae4f22b48c60'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:19.876667 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:19.885037 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a208228c-65a6-4776-aead-ae4f22b48c60 admin admin] The token cache is being invalidate because project a365645db29f4ea4a1bc943669ae9eff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:34:19.885427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a208228c-65a6-4776-aead-ae4f22b48c60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:19.887228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a208228c-65a6-4776-aead-ae4f22b48c60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:19.890447 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 838/1653] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:34:19 2022] DELETE /identity/v3/projects/a365645db29f4ea4a1bc943669ae9eff => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:19.891459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208228c-65a6-4776-aead-ae4f22b48c60 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:19.896067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a208228c-65a6-4776-aead-ae4f22b48c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:19.911915 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-41008cbc-e93e-4c8b-ad50-0a5bc794c7cb None None] MFA Rules not processed for user `c078e655c9454316aa7afe13a6b565e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:19.913445 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-41008cbc-e93e-4c8b-ad50-0a5bc794c7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:20.037172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208228c-65a6-4776-aead-ae4f22b48c60 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:20.053521 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b49013-754f-41b0-b0a7-6b7630cdee2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:20.054442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b49013-754f-41b0-b0a7-6b7630cdee2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:20.055040 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b49013-754f-41b0-b0a7-6b7630cdee2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:20.055512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b49013-754f-41b0-b0a7-6b7630cdee2c admin admin] PATH_INFO: `/v3/projects/108a6003407b4b03ab30834f2af7d66c` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:20.071095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b49013-754f-41b0-b0a7-6b7630cdee2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=108a6003407b4b03ab30834f2af7d66c)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:20.071095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b49013-754f-41b0-b0a7-6b7630cdee2c admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:20.075358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 816/1654] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:34:20.082428 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34d29e-f289-4c11-bd26-38d90e79ef22 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:20.082762 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34d29e-f289-4c11-bd26-38d90e79ef22 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:20.083066 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34d29e-f289-4c11-bd26-38d90e79ef22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:20.126757 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-cf34d29e-f289-4c11-bd26-38d90e79ef22 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:20.129595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-cf34d29e-f289-4c11-bd26-38d90e79ef22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:20.139412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 817/1655] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:20.141334 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34d29e-f289-4c11-bd26-38d90e79ef22 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:20.151780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-cf34d29e-f289-4c11-bd26-38d90e79ef22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:20.246466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e5b49013-754f-41b0-b0a7-6b7630cdee2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '108a6003407b4b03ab30834f2af7d66c', 'request_id': 'req-e5b49013-754f-41b0-b0a7-6b7630cdee2c'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:20.259404 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e5b49013-754f-41b0-b0a7-6b7630cdee2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '108a6003407b4b03ab30834f2af7d66c', 'request_id': 'req-e5b49013-754f-41b0-b0a7-6b7630cdee2c'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:20.305653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34d29e-f289-4c11-bd26-38d90e79ef22 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:20.343261 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:20.345918 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e5b49013-754f-41b0-b0a7-6b7630cdee2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '108a6003407b4b03ab30834f2af7d66c', 'request_id': 'req-e5b49013-754f-41b0-b0a7-6b7630cdee2c'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:20.364133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e5b49013-754f-41b0-b0a7-6b7630cdee2c admin admin] The token cache is being invalidate because project 108a6003407b4b03ab30834f2af7d66c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:34:20.364668 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e5b49013-754f-41b0-b0a7-6b7630cdee2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:20.366679 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e5b49013-754f-41b0-b0a7-6b7630cdee2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:20.368598 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 839/1656] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:34:19 2022] DELETE /identity/v3/projects/108a6003407b4b03ab30834f2af7d66c => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:20.373763 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31a350-d7db-45bc-9a95-6036e1195165 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:20.374243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31a350-d7db-45bc-9a95-6036e1195165 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:20.374678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31a350-d7db-45bc-9a95-6036e1195165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:20.434868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-ec31a350-d7db-45bc-9a95-6036e1195165 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:20.435873 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d9f87-d7ca-402e-98b0-301cb3df977e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:20.436177 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ec31a350-d7db-45bc-9a95-6036e1195165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:20.437948 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d9f87-d7ca-402e-98b0-301cb3df977e service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:20.438664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d9f87-d7ca-402e-98b0-301cb3df977e service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:20.440300 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d9f87-d7ca-402e-98b0-301cb3df977e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:20.444860 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-2c4d9f87-d7ca-402e-98b0-301cb3df977e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:20.532272 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 840/1657] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:20.533711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a77b9-f6ba-4460-b00e-a291afbd1b4b None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:20.534518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a77b9-f6ba-4460-b00e-a291afbd1b4b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:20.534870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a77b9-f6ba-4460-b00e-a291afbd1b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:20.556648 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d9f87-d7ca-402e-98b0-301cb3df977e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:20.557159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-475a77b9-f6ba-4460-b00e-a291afbd1b4b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:20.558899 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d9f87-d7ca-402e-98b0-301cb3df977e service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:20.560156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-475a77b9-f6ba-4460-b00e-a291afbd1b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:20.575759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 841/1658] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:20.575759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a77b9-f6ba-4460-b00e-a291afbd1b4b None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:20.583389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a77b9-f6ba-4460-b00e-a291afbd1b4b None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:20.589926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da482844-0469-49d8-8ede-19ee1392a431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:20.591071 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da482844-0469-49d8-8ede-19ee1392a431 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:20.591506 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da482844-0469-49d8-8ede-19ee1392a431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:20.591794 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da482844-0469-49d8-8ede-19ee1392a431 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:20.592891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da482844-0469-49d8-8ede-19ee1392a431 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:20.593209 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da482844-0469-49d8-8ede-19ee1392a431 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:20.594961 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da482844-0469-49d8-8ede-19ee1392a431 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:20.597264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 818/1659] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:20.599688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64408abe-ae46-4c2a-92cd-de9e2105911f None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:20.602467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64408abe-ae46-4c2a-92cd-de9e2105911f None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:20.602467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64408abe-ae46-4c2a-92cd-de9e2105911f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:20.612490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 842/1660] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:34:20.617031 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da482844-0469-49d8-8ede-19ee1392a431 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:20.644194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da482844-0469-49d8-8ede-19ee1392a431 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:20.660849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5dfbd-be79-4ce1-a060-28acb0973f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:20.660849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5dfbd-be79-4ce1-a060-28acb0973f5f admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:20.660849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5dfbd-be79-4ce1-a060-28acb0973f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:20.661234 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5dfbd-be79-4ce1-a060-28acb0973f5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:20.661718 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5dfbd-be79-4ce1-a060-28acb0973f5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:20.663663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5dfbd-be79-4ce1-a060-28acb0973f5f admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:20.691563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-64408abe-ae46-4c2a-92cd-de9e2105911f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:20.692803 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-64408abe-ae46-4c2a-92cd-de9e2105911f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:20.726626 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 819/1661] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:20.731866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64408abe-ae46-4c2a-92cd-de9e2105911f None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:20.751180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64408abe-ae46-4c2a-92cd-de9e2105911f None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:20.769487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0aedc1-fd96-4879-8b3e-e99d6f44fcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:20.770248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0aedc1-fd96-4879-8b3e-e99d6f44fcf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:20.770630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0aedc1-fd96-4879-8b3e-e99d6f44fcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:20.771023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0aedc1-fd96-4879-8b3e-e99d6f44fcf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:20.771801 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0aedc1-fd96-4879-8b3e-e99d6f44fcf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:20.772204 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0aedc1-fd96-4879-8b3e-e99d6f44fcf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:20.774074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0aedc1-fd96-4879-8b3e-e99d6f44fcf9 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:20.802574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 843/1662] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:20 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:20.804496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5dfbd-be79-4ce1-a060-28acb0973f5f admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:20.808120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 820/1663] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:34:20.811513 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0aedc1-fd96-4879-8b3e-e99d6f44fcf9 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:20.842781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0aedc1-fd96-4879-8b3e-e99d6f44fcf9 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:20.888134 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:21.006857 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5dfbd-be79-4ce1-a060-28acb0973f5f admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:21.009310 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b1743-c97c-4074-be97-4560adcb764e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:21.010333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b1743-c97c-4074-be97-4560adcb764e admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:21.011460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b1743-c97c-4074-be97-4560adcb764e admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:21.011969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b1743-c97c-4074-be97-4560adcb764e admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:21.013132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b1743-c97c-4074-be97-4560adcb764e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:21.019285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3198ab1-5895-447a-a72b-4adbaa518181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:21.020119 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3198ab1-5895-447a-a72b-4adbaa518181 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:21.020523 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3198ab1-5895-447a-a72b-4adbaa518181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:21.020919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3198ab1-5895-447a-a72b-4adbaa518181 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:21.021710 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3198ab1-5895-447a-a72b-4adbaa518181 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:21.025069 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3198ab1-5895-447a-a72b-4adbaa518181 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:21.026291 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b1743-c97c-4074-be97-4560adcb764e admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:21.144627 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 844/1664] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:20 2022] POST /identity/v3/users => generated 287 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:21.149348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3198ab1-5895-447a-a72b-4adbaa518181 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:21.157013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 821/1665] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:20 2022] POST /identity/v3/projects => generated 367 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:21.160942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b1743-c97c-4074-be97-4560adcb764e admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:21.182443 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3198ab1-5895-447a-a72b-4adbaa518181 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:21.270565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:21.343461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15f0fd-c774-4fc2-827c-93e2b4020649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:21.347089 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15f0fd-c774-4fc2-827c-93e2b4020649 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:21.348538 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15f0fd-c774-4fc2-827c-93e2b4020649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:21.348873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15f0fd-c774-4fc2-827c-93e2b4020649 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:21.351782 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15f0fd-c774-4fc2-827c-93e2b4020649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:21.355046 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15f0fd-c774-4fc2-827c-93e2b4020649 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:21.364327 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b1743-c97c-4074-be97-4560adcb764e admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:21.382994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169415e-9c0d-47fd-a6c0-fc5b8c338cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:21.386659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169415e-9c0d-47fd-a6c0-fc5b8c338cfa admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:21.386977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169415e-9c0d-47fd-a6c0-fc5b8c338cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:21.387257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169415e-9c0d-47fd-a6c0-fc5b8c338cfa admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:21.387913 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3169415e-9c0d-47fd-a6c0-fc5b8c338cfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:21.398441 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3169415e-9c0d-47fd-a6c0-fc5b8c338cfa admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:21.405473 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 845/1666] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:21.407813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15f0fd-c774-4fc2-827c-93e2b4020649 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:21.445582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15f0fd-c774-4fc2-827c-93e2b4020649 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:21.461263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d8c77-db2f-4e7f-86b6-2930d11343f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:21.461893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d8c77-db2f-4e7f-86b6-2930d11343f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:21.462179 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d8c77-db2f-4e7f-86b6-2930d11343f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:21.462460 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d8c77-db2f-4e7f-86b6-2930d11343f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:21.463461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d8c77-db2f-4e7f-86b6-2930d11343f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:21.466856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d8c77-db2f-4e7f-86b6-2930d11343f7 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:21.505330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 846/1667] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:21.505806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d8c77-db2f-4e7f-86b6-2930d11343f7 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:21.539323 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 822/1668] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:21 2022] POST /identity/v3/users => generated 284 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:21.541366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d8c77-db2f-4e7f-86b6-2930d11343f7 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:21.542689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169415e-9c0d-47fd-a6c0-fc5b8c338cfa admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:21.555082 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829344b-300d-4e26-98d1-867a743c1793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:21.555777 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829344b-300d-4e26-98d1-867a743c1793 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:21.556060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829344b-300d-4e26-98d1-867a743c1793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:21.556339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829344b-300d-4e26-98d1-867a743c1793 admin admin] PATH_INFO: `/v3/projects/0c9f2cb345994547b81ea276771536cc/users/8f2b43d09c034e07a7378f645a8004c9/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:21.568333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169415e-9c0d-47fd-a6c0-fc5b8c338cfa admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:21.595335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881464e-9c01-4d9e-9d41-c9fbaac2c6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:21.599055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881464e-9c01-4d9e-9d41-c9fbaac2c6ff admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:21.600819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829344b-300d-4e26-98d1-867a743c1793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9f2cb345994547b81ea276771536cc, user_id=8f2b43d09c034e07a7378f645a8004c9, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:21.600947 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881464e-9c01-4d9e-9d41-c9fbaac2c6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:21.600947 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881464e-9c01-4d9e-9d41-c9fbaac2c6ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:21.600947 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881464e-9c01-4d9e-9d41-c9fbaac2c6ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:21.601782 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829344b-300d-4e26-98d1-867a743c1793 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:21.605475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881464e-9c01-4d9e-9d41-c9fbaac2c6ff admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:21.650997 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 823/1669] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:21.652438 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881464e-9c01-4d9e-9d41-c9fbaac2c6ff admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:21.678927 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881464e-9c01-4d9e-9d41-c9fbaac2c6ff admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:21.685452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 847/1670] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:21 2022] PUT /identity/v3/projects/0c9f2cb345994547b81ea276771536cc/users/8f2b43d09c034e07a7378f645a8004c9/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:21.698596 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf23d1-7306-44aa-bd9d-2511675b4bd7 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:21.699364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf23d1-7306-44aa-bd9d-2511675b4bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:21.699656 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf23d1-7306-44aa-bd9d-2511675b4bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:21.778417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-d8bf23d1-7306-44aa-bd9d-2511675b4bd7 None None] MFA Rules not processed for user `8f2b43d09c034e07a7378f645a8004c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:21.781517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d752763-9d67-48bf-b7ad-12c489c90db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:21.782380 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d752763-9d67-48bf-b7ad-12c489c90db2 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:21.782499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d8bf23d1-7306-44aa-bd9d-2511675b4bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:21.782934 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d752763-9d67-48bf-b7ad-12c489c90db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:21.805253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d752763-9d67-48bf-b7ad-12c489c90db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:21.805253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d752763-9d67-48bf-b7ad-12c489c90db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:21.805253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d752763-9d67-48bf-b7ad-12c489c90db2 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:21.818042 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 824/1671] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:21.822608 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d752763-9d67-48bf-b7ad-12c489c90db2 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:21.836182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d752763-9d67-48bf-b7ad-12c489c90db2 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:21.842767 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5a2ee-171a-4d95-916a-7f5a979e9c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:21.843934 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5a2ee-171a-4d95-916a-7f5a979e9c32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:21.844415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5a2ee-171a-4d95-916a-7f5a979e9c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:21.844906 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5a2ee-171a-4d95-916a-7f5a979e9c32 admin admin] PATH_INFO: `/v3/projects/516fa688c57b4dbda180bb54018846d6/users/624b9ced974f476e8ef61dcb89aa631d/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:21.869242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:21.876517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5a2ee-171a-4d95-916a-7f5a979e9c32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=516fa688c57b4dbda180bb54018846d6, user_id=624b9ced974f476e8ef61dcb89aa631d, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:21.882943 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5a2ee-171a-4d95-916a-7f5a979e9c32 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:21.946137 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 848/1672] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:21.947974 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf23d1-7306-44aa-bd9d-2511675b4bd7 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:21.962447 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf23d1-7306-44aa-bd9d-2511675b4bd7 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:21.972730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 825/1673] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:21 2022] PUT /identity/v3/projects/516fa688c57b4dbda180bb54018846d6/users/624b9ced974f476e8ef61dcb89aa631d/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:21.985255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a8078-2fe9-4b8c-9fbb-8bd8092b5506 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:21.985544 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a8078-2fe9-4b8c-9fbb-8bd8092b5506 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:21.985815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a8078-2fe9-4b8c-9fbb-8bd8092b5506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:22.044923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-b47a8078-2fe9-4b8c-9fbb-8bd8092b5506 None None] MFA Rules not processed for user `624b9ced974f476e8ef61dcb89aa631d`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:22.048498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b47a8078-2fe9-4b8c-9fbb-8bd8092b5506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:22.187660 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1421a-7c33-4b9a-be8f-002ee04a5b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:22.189914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1421a-7c33-4b9a-be8f-002ee04a5b0e admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:22.190301 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1421a-7c33-4b9a-be8f-002ee04a5b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:22.191783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1421a-7c33-4b9a-be8f-002ee04a5b0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:22.194135 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1421a-7c33-4b9a-be8f-002ee04a5b0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:22.201013 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1421a-7c33-4b9a-be8f-002ee04a5b0e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:22.220802 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 826/1674] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:22.230289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a8078-2fe9-4b8c-9fbb-8bd8092b5506 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:22.231929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b47a8078-2fe9-4b8c-9fbb-8bd8092b5506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:22.260072 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:22.356163 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a8078-2fe9-4b8c-9fbb-8bd8092b5506 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:22.365592 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c3109-056e-4dcc-8cd9-6cf341155ba0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:22.366309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c3109-056e-4dcc-8cd9-6cf341155ba0 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:22.366597 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c3109-056e-4dcc-8cd9-6cf341155ba0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:22.366907 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c3109-056e-4dcc-8cd9-6cf341155ba0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:22.378741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c3109-056e-4dcc-8cd9-6cf341155ba0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:22.380356 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c3109-056e-4dcc-8cd9-6cf341155ba0 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:22.392964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 827/1675] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:22.406402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 849/1676] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:21 2022] POST /identity/v3/projects => generated 373 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:22.411197 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c3109-056e-4dcc-8cd9-6cf341155ba0 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:22.432288 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c3109-056e-4dcc-8cd9-6cf341155ba0 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:22.569449 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faccb5a-b65a-4ef4-88d4-6b1736e71f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:22.570163 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faccb5a-b65a-4ef4-88d4-6b1736e71f7f admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:22.572094 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faccb5a-b65a-4ef4-88d4-6b1736e71f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:22.572379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faccb5a-b65a-4ef4-88d4-6b1736e71f7f admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:22.574198 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faccb5a-b65a-4ef4-88d4-6b1736e71f7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:22.578360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faccb5a-b65a-4ef4-88d4-6b1736e71f7f admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:22.719765 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 828/1677] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:22 2022] POST /identity/v3/users => generated 293 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:22.720455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1421a-7c33-4b9a-be8f-002ee04a5b0e admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:22.745833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1421a-7c33-4b9a-be8f-002ee04a5b0e admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:22.766442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f199b0-e1fc-4efb-8451-bb7b634132d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:22.767204 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f199b0-e1fc-4efb-8451-bb7b634132d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:22.767557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f199b0-e1fc-4efb-8451-bb7b634132d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:22.768015 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f199b0-e1fc-4efb-8451-bb7b634132d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:22.769362 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f199b0-e1fc-4efb-8451-bb7b634132d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:22.776889 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f199b0-e1fc-4efb-8451-bb7b634132d0 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:22.790967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 850/1678] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:34:22.794957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faccb5a-b65a-4ef4-88d4-6b1736e71f7f admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:22.806185 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faccb5a-b65a-4ef4-88d4-6b1736e71f7f admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:22.811354 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9082de-eeff-4a77-b267-0fe920a255ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:22.812203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9082de-eeff-4a77-b267-0fe920a255ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:22.812367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9082de-eeff-4a77-b267-0fe920a255ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:22.814057 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9082de-eeff-4a77-b267-0fe920a255ba admin admin] PATH_INFO: `/v3/projects/41d27f1905194b70abb6d0b2ea922d8a/users/11c431818ade47ebb256f33011e75c2c/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:22.833927 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9082de-eeff-4a77-b267-0fe920a255ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=41d27f1905194b70abb6d0b2ea922d8a, user_id=11c431818ade47ebb256f33011e75c2c, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:22.838458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9082de-eeff-4a77-b267-0fe920a255ba admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:22.980532 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 829/1679] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:22 2022] PUT /identity/v3/projects/41d27f1905194b70abb6d0b2ea922d8a/users/11c431818ade47ebb256f33011e75c2c/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:22.985158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f199b0-e1fc-4efb-8451-bb7b634132d0 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:23.004500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f199b0-e1fc-4efb-8451-bb7b634132d0 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:23.138749 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b37ef-d59c-4e81-95d8-dd30f1cdddac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:23.140021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b37ef-d59c-4e81-95d8-dd30f1cdddac admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:23.140662 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b37ef-d59c-4e81-95d8-dd30f1cdddac admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:23.141301 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b37ef-d59c-4e81-95d8-dd30f1cdddac admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:23.142671 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b37ef-d59c-4e81-95d8-dd30f1cdddac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:23.145788 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b37ef-d59c-4e81-95d8-dd30f1cdddac admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:23.164170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 851/1680] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:23.168575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9082de-eeff-4a77-b267-0fe920a255ba admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:23.183546 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9082de-eeff-4a77-b267-0fe920a255ba admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:23.200990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306dc110-c8f5-48a6-8139-24e527f3aa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:23.201869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dc110-c8f5-48a6-8139-24e527f3aa0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:23.202158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dc110-c8f5-48a6-8139-24e527f3aa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:23.202437 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dc110-c8f5-48a6-8139-24e527f3aa0b admin admin] PATH_INFO: `/v3/projects/41d27f1905194b70abb6d0b2ea922d8a/users/11c431818ade47ebb256f33011e75c2c/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:23.204557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306dc110-c8f5-48a6-8139-24e527f3aa0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=41d27f1905194b70abb6d0b2ea922d8a, user_id=11c431818ade47ebb256f33011e75c2c, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:23.208852 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306dc110-c8f5-48a6-8139-24e527f3aa0b admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:23.273337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cfc3b-0290-4134-b4b9-e9107246cc1b None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:23.273626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cfc3b-0290-4134-b4b9-e9107246cc1b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:23.273956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cfc3b-0290-4134-b4b9-e9107246cc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:23.276682 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 830/1681] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:23 2022] PUT /identity/v3/projects/41d27f1905194b70abb6d0b2ea922d8a/users/11c431818ade47ebb256f33011e75c2c/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:23.306608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-918cfc3b-0290-4134-b4b9-e9107246cc1b None None] MFA Rules not processed for user `11c431818ade47ebb256f33011e75c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:23.308340 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-918cfc3b-0290-4134-b4b9-e9107246cc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:23.349638 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:23.444305 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 852/1682] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:23.464495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306dc110-c8f5-48a6-8139-24e527f3aa0b admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:23.559335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fa9fb-9683-4324-bfa3-1a99916cb0e0 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:23.559729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fa9fb-9683-4324-bfa3-1a99916cb0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:23.559729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fa9fb-9683-4324-bfa3-1a99916cb0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:23.581690 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306dc110-c8f5-48a6-8139-24e527f3aa0b admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:23.600877 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a49e6f-016e-4c16-8b07-d0e42b1f0adc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:23.602037 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a49e6f-016e-4c16-8b07-d0e42b1f0adc service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:23.602037 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a49e6f-016e-4c16-8b07-d0e42b1f0adc service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:23.602277 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a49e6f-016e-4c16-8b07-d0e42b1f0adc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:23.615172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-a62fa9fb-9683-4324-bfa3-1a99916cb0e0 None None] MFA Rules not processed for user `9136f02e6c664f27b3f4fd35475f0eea`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:23.616700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a62fa9fb-9683-4324-bfa3-1a99916cb0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:23.632333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a49e6f-016e-4c16-8b07-d0e42b1f0adc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:23.634502 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a49e6f-016e-4c16-8b07-d0e42b1f0adc service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:23.651593 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 831/1683] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:23.777653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 853/1684] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:23.779413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14b444-9934-4d4a-8626-4dfd6c121531 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:23.779751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14b444-9934-4d4a-8626-4dfd6c121531 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:23.781071 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14b444-9934-4d4a-8626-4dfd6c121531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:23.824044 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-6f14b444-9934-4d4a-8626-4dfd6c121531 None None] MFA Rules not processed for user `9136f02e6c664f27b3f4fd35475f0eea`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:23.834530 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-6f14b444-9934-4d4a-8626-4dfd6c121531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:23.856539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 832/1685] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:23.863244 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fa9fb-9683-4324-bfa3-1a99916cb0e0 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:23.864679 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a62fa9fb-9683-4324-bfa3-1a99916cb0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:24.017487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fa9fb-9683-4324-bfa3-1a99916cb0e0 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:24.026232 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d442717-6f6a-491e-81a8-f520c0c5b83e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:24.026911 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d442717-6f6a-491e-81a8-f520c0c5b83e service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:24.027242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d442717-6f6a-491e-81a8-f520c0c5b83e service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:24.027465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d442717-6f6a-491e-81a8-f520c0c5b83e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:24.038798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d442717-6f6a-491e-81a8-f520c0c5b83e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:24.040727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d442717-6f6a-491e-81a8-f520c0c5b83e service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:24.040986 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14b444-9934-4d4a-8626-4dfd6c121531 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:24.042458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-6f14b444-9934-4d4a-8626-4dfd6c121531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:24.048022 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:24.058659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 854/1686] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:24.059613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d442717-6f6a-491e-81a8-f520c0c5b83e service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:24.069608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d442717-6f6a-491e-81a8-f520c0c5b83e service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:24.096687 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483286d0-5509-4698-b137-ce1fcf26a5f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:24.125743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483286d0-5509-4698-b137-ce1fcf26a5f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:24.126030 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483286d0-5509-4698-b137-ce1fcf26a5f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:24.129723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483286d0-5509-4698-b137-ce1fcf26a5f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:24.264954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14b444-9934-4d4a-8626-4dfd6c121531 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:24.291561 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e36f69-dbea-41bf-8a40-4192b7db2f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:24.292607 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e36f69-dbea-41bf-8a40-4192b7db2f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:24.292895 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e36f69-dbea-41bf-8a40-4192b7db2f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:24.293169 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e36f69-dbea-41bf-8a40-4192b7db2f54 admin admin] PATH_INFO: `/v3/users/87e458756a534cb38c823e884b3d7569` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:24.296027 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e36f69-dbea-41bf-8a40-4192b7db2f54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87e458756a534cb38c823e884b3d7569)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:24.299844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e36f69-dbea-41bf-8a40-4192b7db2f54 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:24.422467 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483286d0-5509-4698-b137-ce1fcf26a5f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:24.422467 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483286d0-5509-4698-b137-ce1fcf26a5f3 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:24.451700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 855/1687] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:24.587617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-90e36f69-dbea-41bf-8a40-4192b7db2f54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87e458756a534cb38c823e884b3d7569'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:24.668164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-90e36f69-dbea-41bf-8a40-4192b7db2f54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87e458756a534cb38c823e884b3d7569'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:24.668961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-90e36f69-dbea-41bf-8a40-4192b7db2f54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87e458756a534cb38c823e884b3d7569'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:24.701882 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-90e36f69-dbea-41bf-8a40-4192b7db2f54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87e458756a534cb38c823e884b3d7569'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:24.731804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-90e36f69-dbea-41bf-8a40-4192b7db2f54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87e458756a534cb38c823e884b3d7569'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:24.740845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 833/1688] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:34:24 2022] DELETE /identity/v3/users/87e458756a534cb38c823e884b3d7569 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:24.745382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b80c75-10e2-46d7-8387-e7fc7ab3256f None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:24.745746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b80c75-10e2-46d7-8387-e7fc7ab3256f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:24.746100 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b80c75-10e2-46d7-8387-e7fc7ab3256f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:24.807868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-44b80c75-10e2-46d7-8387-e7fc7ab3256f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:24.811685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-44b80c75-10e2-46d7-8387-e7fc7ab3256f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:24.954124 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:24.983124 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 856/1689] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:25.023395 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e36f69-dbea-41bf-8a40-4192b7db2f54 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:25.025118 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-90e36f69-dbea-41bf-8a40-4192b7db2f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:25.160285 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e36f69-dbea-41bf-8a40-4192b7db2f54 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:25.182692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd3d02-2054-4586-b655-16492befb1e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:25.183837 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd3d02-2054-4586-b655-16492befb1e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:25.185242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd3d02-2054-4586-b655-16492befb1e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:25.185701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd3d02-2054-4586-b655-16492befb1e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:25.201873 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd3d02-2054-4586-b655-16492befb1e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:25.203522 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd3d02-2054-4586-b655-16492befb1e7 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:25.243193 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 834/1690] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:25.373995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f913d26-3880-48ff-a08b-506e0aff4552 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:25.374832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f913d26-3880-48ff-a08b-506e0aff4552 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:25.375402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f913d26-3880-48ff-a08b-506e0aff4552 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:25.378122 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 857/1691] 10.222.0.15 () {54 vars in 912 bytes} [Mon Sep 5 14:34:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:34:25.384548 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6ca75-b73a-44c7-907d-64cd81d93a9c None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:25.385072 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6ca75-b73a-44c7-907d-64cd81d93a9c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:25.385310 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6ca75-b73a-44c7-907d-64cd81d93a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:25.480063 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-57a6ca75-b73a-44c7-907d-64cd81d93a9c None None] MFA Rules not processed for user `5c73ca2785334fc496b5227d3be655c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:25.481559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-57a6ca75-b73a-44c7-907d-64cd81d93a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:25.503852 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 835/1692] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 5 14:34:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:25.508580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f913d26-3880-48ff-a08b-506e0aff4552 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:25.531267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f913d26-3880-48ff-a08b-506e0aff4552 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:25.557197 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3cdd8-b691-41c3-b04b-595b05ef3062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:25.557615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3cdd8-b691-41c3-b04b-595b05ef3062 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:25.557896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3cdd8-b691-41c3-b04b-595b05ef3062 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:25.572517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3cdd8-b691-41c3-b04b-595b05ef3062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:25.610205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3cdd8-b691-41c3-b04b-595b05ef3062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:25.617835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3cdd8-b691-41c3-b04b-595b05ef3062 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:25.653076 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 858/1693] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:25.753755 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231f8c0-b67d-4402-bd2d-e6460e1edaef None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:25.753981 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231f8c0-b67d-4402-bd2d-e6460e1edaef None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:25.754262 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231f8c0-b67d-4402-bd2d-e6460e1edaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:25.765869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:25.814477 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-f231f8c0-b67d-4402-bd2d-e6460e1edaef None None] MFA Rules not processed for user `91c691b6a7724d708bfc7424908516c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:25.815794 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f231f8c0-b67d-4402-bd2d-e6460e1edaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:26.002086 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 836/1694] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:26.016553 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3cdd8-b691-41c3-b04b-595b05ef3062 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:26.019241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-71b3cdd8-b691-41c3-b04b-595b05ef3062 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:26.152578 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3cdd8-b691-41c3-b04b-595b05ef3062 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:26.159371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ff44d-2418-4549-a22d-a6b47b9a70e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:26.160250 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ff44d-2418-4549-a22d-a6b47b9a70e9 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:26.160688 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ff44d-2418-4549-a22d-a6b47b9a70e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:26.161116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ff44d-2418-4549-a22d-a6b47b9a70e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:26.187925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ff44d-2418-4549-a22d-a6b47b9a70e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:26.189546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ff44d-2418-4549-a22d-a6b47b9a70e9 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:26.215534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 859/1695] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:26.461367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231f8c0-b67d-4402-bd2d-e6460e1edaef None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:26.462144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f231f8c0-b67d-4402-bd2d-e6460e1edaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:26.501795 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231f8c0-b67d-4402-bd2d-e6460e1edaef None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:26.518472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c390c6-ba8a-4883-ab13-944097802d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:26.518472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c390c6-ba8a-4883-ab13-944097802d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:26.518472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c390c6-ba8a-4883-ab13-944097802d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:26.518472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c390c6-ba8a-4883-ab13-944097802d76 admin admin] PATH_INFO: `/v3/projects/31e97d66fc9d4f51ac0849d5ee258907` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:26.520213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c390c6-ba8a-4883-ab13-944097802d76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e97d66fc9d4f51ac0849d5ee258907)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:26.526333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c390c6-ba8a-4883-ab13-944097802d76 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:26.759379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f4c390c6-ba8a-4883-ab13-944097802d76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e97d66fc9d4f51ac0849d5ee258907', 'request_id': 'req-f4c390c6-ba8a-4883-ab13-944097802d76'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:26.792362 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ff44d-2418-4549-a22d-a6b47b9a70e9 service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:26.804443 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-826ff44d-2418-4549-a22d-a6b47b9a70e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:26.806407 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f4c390c6-ba8a-4883-ab13-944097802d76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e97d66fc9d4f51ac0849d5ee258907', 'request_id': 'req-f4c390c6-ba8a-4883-ab13-944097802d76'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:26.894078 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:26.903641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f4c390c6-ba8a-4883-ab13-944097802d76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e97d66fc9d4f51ac0849d5ee258907', 'request_id': 'req-f4c390c6-ba8a-4883-ab13-944097802d76'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:26.929238 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f4c390c6-ba8a-4883-ab13-944097802d76 admin admin] The token cache is being invalidate because project 31e97d66fc9d4f51ac0849d5ee258907 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:34:26.933756 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f4c390c6-ba8a-4883-ab13-944097802d76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:26.936610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f4c390c6-ba8a-4883-ab13-944097802d76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:26.947070 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 837/1696] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:34:26 2022] DELETE /identity/v3/projects/31e97d66fc9d4f51ac0849d5ee258907 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:26.951393 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ff44d-2418-4549-a22d-a6b47b9a70e9 service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:26.997355 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c390c6-ba8a-4883-ab13-944097802d76 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:27.011711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f4c390c6-ba8a-4883-ab13-944097802d76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:27.201953 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f191470-b488-4e7d-a2d8-c0550a41bc1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:27.205704 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f191470-b488-4e7d-a2d8-c0550a41bc1b service placement] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:27.208123 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f191470-b488-4e7d-a2d8-c0550a41bc1b service placement] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:27.212390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f191470-b488-4e7d-a2d8-c0550a41bc1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:27.212390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8f191470-b488-4e7d-a2d8-c0550a41bc1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:27.228762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f191470-b488-4e7d-a2d8-c0550a41bc1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:27.230852 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f191470-b488-4e7d-a2d8-c0550a41bc1b service placement] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:27.268171 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 860/1697] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:34:27.270840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d53bb4-2775-47c8-b0ab-0cbf9c12c062 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:27.273747 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d53bb4-2775-47c8-b0ab-0cbf9c12c062 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:27.274201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d53bb4-2775-47c8-b0ab-0cbf9c12c062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:27.334832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c390c6-ba8a-4883-ab13-944097802d76 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:27.342172 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7fc62-c76a-40e7-87ca-bfacd9476d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:27.343967 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7fc62-c76a-40e7-87ca-bfacd9476d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:27.345264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7fc62-c76a-40e7-87ca-bfacd9476d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:27.346696 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7fc62-c76a-40e7-87ca-bfacd9476d2a admin admin] PATH_INFO: `/v3/users/b8ecac515c894105818cde6bfcc32542` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:27.359144 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-07d53bb4-2775-47c8-b0ab-0cbf9c12c062 None None] MFA Rules not processed for user `d31427f2a67e49d0b4d3e1b3cee8c48b`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:27.359648 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7fc62-c76a-40e7-87ca-bfacd9476d2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8ecac515c894105818cde6bfcc32542)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:27.360902 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-07d53bb4-2775-47c8-b0ab-0cbf9c12c062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:27.364511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7fc62-c76a-40e7-87ca-bfacd9476d2a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:27.494356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 861/1698] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:27.494356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e62729-a0cd-4e1e-8616-6d068181d854 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:27.494356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e62729-a0cd-4e1e-8616-6d068181d854 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:27.494356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e62729-a0cd-4e1e-8616-6d068181d854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:27.519352 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-32e62729-a0cd-4e1e-8616-6d068181d854 None None] MFA Rules not processed for user `9136f02e6c664f27b3f4fd35475f0eea`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:27.522150 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-32e62729-a0cd-4e1e-8616-6d068181d854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:27.655039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-03f7fc62-c76a-40e7-87ca-bfacd9476d2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8ecac515c894105818cde6bfcc32542'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:27.660769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 862/1699] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:27.665557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e62729-a0cd-4e1e-8616-6d068181d854 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:27.667110 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-32e62729-a0cd-4e1e-8616-6d068181d854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:27.720185 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e62729-a0cd-4e1e-8616-6d068181d854 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:27.726394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-03f7fc62-c76a-40e7-87ca-bfacd9476d2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8ecac515c894105818cde6bfcc32542'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:27.729091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-03f7fc62-c76a-40e7-87ca-bfacd9476d2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8ecac515c894105818cde6bfcc32542'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:27.746929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-03f7fc62-c76a-40e7-87ca-bfacd9476d2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8ecac515c894105818cde6bfcc32542'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:27.760849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8ab50-346f-46e3-8b3b-b3e9d97c3280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:27.761374 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8ab50-346f-46e3-8b3b-b3e9d97c3280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:27.761772 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8ab50-346f-46e3-8b3b-b3e9d97c3280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:27.762058 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8ab50-346f-46e3-8b3b-b3e9d97c3280 admin admin] PATH_INFO: `/v3/users/bbb6b8c6d8d04e72a1fc06dfa2bb3692` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:27.764593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf8ab50-346f-46e3-8b3b-b3e9d97c3280 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbb6b8c6d8d04e72a1fc06dfa2bb3692)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:27.768874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-03f7fc62-c76a-40e7-87ca-bfacd9476d2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8ecac515c894105818cde6bfcc32542'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:27.769494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf8ab50-346f-46e3-8b3b-b3e9d97c3280 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:27.783714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 838/1700] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:34:26 2022] DELETE /identity/v3/users/b8ecac515c894105818cde6bfcc32542 => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:27.786283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e984e7-e134-4ff3-9f87-3741caac35b1 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:27.786762 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e984e7-e134-4ff3-9f87-3741caac35b1 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:27.791756 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e984e7-e134-4ff3-9f87-3741caac35b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:27.827025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-31e984e7-e134-4ff3-9f87-3741caac35b1 None None] MFA Rules not processed for user `d31427f2a67e49d0b4d3e1b3cee8c48b`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:27.828832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-31e984e7-e134-4ff3-9f87-3741caac35b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:27.856347 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:28.034945 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:28.091415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 839/1701] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:28.105672 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e984e7-e134-4ff3-9f87-3741caac35b1 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:28.107914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-31e984e7-e134-4ff3-9f87-3741caac35b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:28.166149 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-2bf8ab50-346f-46e3-8b3b-b3e9d97c3280 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbb6b8c6d8d04e72a1fc06dfa2bb3692'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:28.236910 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-2bf8ab50-346f-46e3-8b3b-b3e9d97c3280 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbb6b8c6d8d04e72a1fc06dfa2bb3692'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:28.303662 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-2bf8ab50-346f-46e3-8b3b-b3e9d97c3280 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbb6b8c6d8d04e72a1fc06dfa2bb3692'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:28.308583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-2bf8ab50-346f-46e3-8b3b-b3e9d97c3280 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbb6b8c6d8d04e72a1fc06dfa2bb3692'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:28.341148 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-2bf8ab50-346f-46e3-8b3b-b3e9d97c3280 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbb6b8c6d8d04e72a1fc06dfa2bb3692'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:28.373854 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 863/1702] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:34:27 2022] DELETE /identity/v3/users/bbb6b8c6d8d04e72a1fc06dfa2bb3692 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:28.379185 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f0ac0-bc71-409e-bd14-8caa84698356 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:28.379678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f0ac0-bc71-409e-bd14-8caa84698356 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:28.380442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f0ac0-bc71-409e-bd14-8caa84698356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:28.406025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e984e7-e134-4ff3-9f87-3741caac35b1 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:28.466743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-8d8f0ac0-bc71-409e-bd14-8caa84698356 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:28.471526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8d8f0ac0-bc71-409e-bd14-8caa84698356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:28.629347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc4f2e-8b1f-4035-81f6-62d71acc0296 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:28.629990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc4f2e-8b1f-4035-81f6-62d71acc0296 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:28.630614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc4f2e-8b1f-4035-81f6-62d71acc0296 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:28.631192 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc4f2e-8b1f-4035-81f6-62d71acc0296 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:28.636659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-90dc4f2e-8b1f-4035-81f6-62d71acc0296 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:28.671554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 864/1703] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:28.743998 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f0ac0-bc71-409e-bd14-8caa84698356 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:28.745878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8d8f0ac0-bc71-409e-bd14-8caa84698356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:28.837775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc4f2e-8b1f-4035-81f6-62d71acc0296 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:28.839697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc4f2e-8b1f-4035-81f6-62d71acc0296 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:28.882441 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 840/1704] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 777 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:28.885249 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6981f-28da-4e13-aefa-99c523b08049 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:28.885541 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6981f-28da-4e13-aefa-99c523b08049 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:28.887468 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6981f-28da-4e13-aefa-99c523b08049 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:28.906120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 841/1705] 10.222.0.15 () {54 vars in 912 bytes} [Mon Sep 5 14:34:28 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:34:28.911144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69895718-3461-4c40-9ccf-7097ec5e3e56 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:28.911433 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69895718-3461-4c40-9ccf-7097ec5e3e56 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:28.911890 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69895718-3461-4c40-9ccf-7097ec5e3e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:28.912873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f0ac0-bc71-409e-bd14-8caa84698356 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:28.920721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4a7d2-f17a-4c6d-8e73-e3c6ec69d863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:28.921486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4a7d2-f17a-4c6d-8e73-e3c6ec69d863 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:28.921894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4a7d2-f17a-4c6d-8e73-e3c6ec69d863 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:28.922262 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4a7d2-f17a-4c6d-8e73-e3c6ec69d863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:28.941922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4a7d2-f17a-4c6d-8e73-e3c6ec69d863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:28.944362 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4a7d2-f17a-4c6d-8e73-e3c6ec69d863 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:28.968136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-69895718-3461-4c40-9ccf-7097ec5e3e56 None None] MFA Rules not processed for user `5c73ca2785334fc496b5227d3be655c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:28.969612 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 865/1706] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:28.975895 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-69895718-3461-4c40-9ccf-7097ec5e3e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:29.007068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 842/1707] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 5 14:34:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:34:29.089622 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4a7d2-f17a-4c6d-8e73-e3c6ec69d863 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:29.110243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4a7d2-f17a-4c6d-8e73-e3c6ec69d863 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:29.119307 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e5a1c-7742-4205-8a34-3ef7a9269104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:29.120164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e5a1c-7742-4205-8a34-3ef7a9269104 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:29.120651 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e5a1c-7742-4205-8a34-3ef7a9269104 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:29.121058 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e5a1c-7742-4205-8a34-3ef7a9269104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:29.156752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e5a1c-7742-4205-8a34-3ef7a9269104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:29.159061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e5a1c-7742-4205-8a34-3ef7a9269104 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:29.176698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 866/1708] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:29.482656 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8bc18-bd23-40e4-a7e1-0f6e3c6d6954 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:29.483301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8bc18-bd23-40e4-a7e1-0f6e3c6d6954 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:29.483676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8bc18-bd23-40e4-a7e1-0f6e3c6d6954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:29.521203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-dae8bc18-bd23-40e4-a7e1-0f6e3c6d6954 None None] MFA Rules not processed for user `c078e655c9454316aa7afe13a6b565e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:29.522687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-dae8bc18-bd23-40e4-a7e1-0f6e3c6d6954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:29.653641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 843/1709] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:30.161511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e5a1c-7742-4205-8a34-3ef7a9269104 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:30.163818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7c0e5a1c-7742-4205-8a34-3ef7a9269104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:30.189897 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e5a1c-7742-4205-8a34-3ef7a9269104 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:30.203447 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f87458-8fbe-4c64-a81f-90a12ac11923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:30.204486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f87458-8fbe-4c64-a81f-90a12ac11923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:30.206092 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f87458-8fbe-4c64-a81f-90a12ac11923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:30.206517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f87458-8fbe-4c64-a81f-90a12ac11923 admin admin] PATH_INFO: `/v3/projects/33a94d38f1be4967b127233847b0fc28` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:30.209511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f87458-8fbe-4c64-a81f-90a12ac11923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33a94d38f1be4967b127233847b0fc28)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:30.214001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f87458-8fbe-4c64-a81f-90a12ac11923 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:30.226681 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:30.295174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:30.417085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-76f87458-8fbe-4c64-a81f-90a12ac11923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33a94d38f1be4967b127233847b0fc28', 'request_id': 'req-76f87458-8fbe-4c64-a81f-90a12ac11923'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:30.428076 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-76f87458-8fbe-4c64-a81f-90a12ac11923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33a94d38f1be4967b127233847b0fc28', 'request_id': 'req-76f87458-8fbe-4c64-a81f-90a12ac11923'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:30.537018 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-76f87458-8fbe-4c64-a81f-90a12ac11923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33a94d38f1be4967b127233847b0fc28', 'request_id': 'req-76f87458-8fbe-4c64-a81f-90a12ac11923'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:30.561189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-76f87458-8fbe-4c64-a81f-90a12ac11923 admin admin] The token cache is being invalidate because project 33a94d38f1be4967b127233847b0fc28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:34:30.561474 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-76f87458-8fbe-4c64-a81f-90a12ac11923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:30.565550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-76f87458-8fbe-4c64-a81f-90a12ac11923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:30.570883 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 867/1710] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:34:30 2022] DELETE /identity/v3/projects/33a94d38f1be4967b127233847b0fc28 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:30.577404 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea304b94-6587-459f-a1cd-e643794be1eb None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:30.577404 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea304b94-6587-459f-a1cd-e643794be1eb None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:30.577404 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea304b94-6587-459f-a1cd-e643794be1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:30.611375 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-ea304b94-6587-459f-a1cd-e643794be1eb None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:30.611375 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ea304b94-6587-459f-a1cd-e643794be1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:30.669196 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f87458-8fbe-4c64-a81f-90a12ac11923 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:30.670159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-76f87458-8fbe-4c64-a81f-90a12ac11923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:30.714999 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 844/1711] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:30.724457 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055dcd5-eb42-4f3a-8bb9-f668ad6782c1 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:30.724904 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055dcd5-eb42-4f3a-8bb9-f668ad6782c1 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:30.725192 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055dcd5-eb42-4f3a-8bb9-f668ad6782c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:30.843099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-c055dcd5-eb42-4f3a-8bb9-f668ad6782c1 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:30.851392 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c055dcd5-eb42-4f3a-8bb9-f668ad6782c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:30.883306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 845/1712] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:30.886351 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055dcd5-eb42-4f3a-8bb9-f668ad6782c1 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:30.916446 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055dcd5-eb42-4f3a-8bb9-f668ad6782c1 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:30.932610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7236d0c-7b7a-4bbc-af92-aec614acd60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:30.933628 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7236d0c-7b7a-4bbc-af92-aec614acd60c admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:30.934055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7236d0c-7b7a-4bbc-af92-aec614acd60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:30.934565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7236d0c-7b7a-4bbc-af92-aec614acd60c admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:30.936341 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7236d0c-7b7a-4bbc-af92-aec614acd60c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:30.939446 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7236d0c-7b7a-4bbc-af92-aec614acd60c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:30.944060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7236d0c-7b7a-4bbc-af92-aec614acd60c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:30.978503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 846/1713] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:34:30.981212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7236d0c-7b7a-4bbc-af92-aec614acd60c admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:31.011194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7236d0c-7b7a-4bbc-af92-aec614acd60c admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:31.022436 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f87458-8fbe-4c64-a81f-90a12ac11923 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:31.029298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9d634-ac71-4a65-bb4f-24982e85d331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:31.030303 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9d634-ac71-4a65-bb4f-24982e85d331 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:31.030766 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9d634-ac71-4a65-bb4f-24982e85d331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:31.031600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9d634-ac71-4a65-bb4f-24982e85d331 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:31.032860 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c9d634-ac71-4a65-bb4f-24982e85d331 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:31.036324 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c9d634-ac71-4a65-bb4f-24982e85d331 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:31.058404 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74ce0a-54f2-4902-acf5-1f4b84c6850a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:31.064336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74ce0a-54f2-4902-acf5-1f4b84c6850a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:31.065233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74ce0a-54f2-4902-acf5-1f4b84c6850a admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:31.068095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74ce0a-54f2-4902-acf5-1f4b84c6850a admin admin] PATH_INFO: `/v3/projects/11dfac0c6f8f47b9a02dbcc53db203e4` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:31.087759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74ce0a-54f2-4902-acf5-1f4b84c6850a admin admin] RBAC: Authorizing `identity:delete_project(project_id=11dfac0c6f8f47b9a02dbcc53db203e4)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:31.112609 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74ce0a-54f2-4902-acf5-1f4b84c6850a admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:31.248960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 847/1714] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:30 2022] POST /identity/v3/projects => generated 371 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:31.256122 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9d634-ac71-4a65-bb4f-24982e85d331 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:31.340711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-de74ce0a-54f2-4902-acf5-1f4b84c6850a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11dfac0c6f8f47b9a02dbcc53db203e4', 'request_id': 'req-de74ce0a-54f2-4902-acf5-1f4b84c6850a'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:31.351832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-de74ce0a-54f2-4902-acf5-1f4b84c6850a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11dfac0c6f8f47b9a02dbcc53db203e4', 'request_id': 'req-de74ce0a-54f2-4902-acf5-1f4b84c6850a'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:31.445966 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-de74ce0a-54f2-4902-acf5-1f4b84c6850a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11dfac0c6f8f47b9a02dbcc53db203e4', 'request_id': 'req-de74ce0a-54f2-4902-acf5-1f4b84c6850a'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:31.450263 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9d634-ac71-4a65-bb4f-24982e85d331 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:31.481817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-de74ce0a-54f2-4902-acf5-1f4b84c6850a admin admin] The token cache is being invalidate because project 11dfac0c6f8f47b9a02dbcc53db203e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:34:31.484283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-de74ce0a-54f2-4902-acf5-1f4b84c6850a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:31.489589 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-de74ce0a-54f2-4902-acf5-1f4b84c6850a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:31.494906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 868/1715] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:34:30 2022] DELETE /identity/v3/projects/11dfac0c6f8f47b9a02dbcc53db203e4 => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:31.500433 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe6ffe-bfc6-4ce0-802f-66fb62320b78 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:31.501236 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe6ffe-bfc6-4ce0-802f-66fb62320b78 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:31.501681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe6ffe-bfc6-4ce0-802f-66fb62320b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:31.543730 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-37fe6ffe-bfc6-4ce0-802f-66fb62320b78 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:31.545266 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-37fe6ffe-bfc6-4ce0-802f-66fb62320b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:31.582595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe928288-82a2-4c0b-95b9-49974cdf9cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:31.583613 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe928288-82a2-4c0b-95b9-49974cdf9cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:31.584025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe928288-82a2-4c0b-95b9-49974cdf9cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:31.584408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe928288-82a2-4c0b-95b9-49974cdf9cd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:31.587480 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe928288-82a2-4c0b-95b9-49974cdf9cd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:31.590406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe928288-82a2-4c0b-95b9-49974cdf9cd0 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:31.647786 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 869/1716] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:31.649504 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6be86-6ffd-4346-afe4-dabe7bf97ea0 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:31.649841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6be86-6ffd-4346-afe4-dabe7bf97ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:31.650070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6be86-6ffd-4346-afe4-dabe7bf97ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:31.687911 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-61e6be86-6ffd-4346-afe4-dabe7bf97ea0 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:31.689168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-61e6be86-6ffd-4346-afe4-dabe7bf97ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:31.711410 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 848/1717] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:31 2022] POST /identity/v3/users => generated 286 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:31.715435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe928288-82a2-4c0b-95b9-49974cdf9cd0 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:31.720001 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-fe928288-82a2-4c0b-95b9-49974cdf9cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:31.733034 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 870/1718] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:34:31.744655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6be86-6ffd-4346-afe4-dabe7bf97ea0 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:31.783774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe928288-82a2-4c0b-95b9-49974cdf9cd0 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:31.795619 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3cd7e9-f3aa-4a6d-b39f-2ac96caf50d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:31.796621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cd7e9-f3aa-4a6d-b39f-2ac96caf50d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:31.798443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cd7e9-f3aa-4a6d-b39f-2ac96caf50d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:31.799459 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cd7e9-f3aa-4a6d-b39f-2ac96caf50d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:31.802081 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6be86-6ffd-4346-afe4-dabe7bf97ea0 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:31.802203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3cd7e9-f3aa-4a6d-b39f-2ac96caf50d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:31.804443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3cd7e9-f3aa-4a6d-b39f-2ac96caf50d5 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:31.832865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc05cf-b7c1-484e-b7fb-0a4dcb79a000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:31.832865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc05cf-b7c1-484e-b7fb-0a4dcb79a000 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:31.832865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc05cf-b7c1-484e-b7fb-0a4dcb79a000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:31.832865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc05cf-b7c1-484e-b7fb-0a4dcb79a000 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:31.833214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc05cf-b7c1-484e-b7fb-0a4dcb79a000 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:31.833214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc05cf-b7c1-484e-b7fb-0a4dcb79a000 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:31.833357 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 849/1719] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:31.833981 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc05cf-b7c1-484e-b7fb-0a4dcb79a000 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:31.840820 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3cd7e9-f3aa-4a6d-b39f-2ac96caf50d5 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:31.853583 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3cd7e9-f3aa-4a6d-b39f-2ac96caf50d5 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:31.858566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbadd3-1280-4bf7-8bfa-1ef299843d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:31.859218 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbadd3-1280-4bf7-8bfa-1ef299843d1f admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:31.859500 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbadd3-1280-4bf7-8bfa-1ef299843d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:31.861452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbadd3-1280-4bf7-8bfa-1ef299843d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:31.861452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becbadd3-1280-4bf7-8bfa-1ef299843d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:31.862898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becbadd3-1280-4bf7-8bfa-1ef299843d1f admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:31.876153 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 871/1720] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:34:31.878194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc05cf-b7c1-484e-b7fb-0a4dcb79a000 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:31.890056 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 850/1721] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:31.890056 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbadd3-1280-4bf7-8bfa-1ef299843d1f admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:31.901946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbadd3-1280-4bf7-8bfa-1ef299843d1f admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:31.914337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc05cf-b7c1-484e-b7fb-0a4dcb79a000 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:31.922156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f458f1-d2e1-454b-90fd-fee3b4ff2fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:31.923119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f458f1-d2e1-454b-90fd-fee3b4ff2fbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:31.923408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f458f1-d2e1-454b-90fd-fee3b4ff2fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:31.923688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f458f1-d2e1-454b-90fd-fee3b4ff2fbe admin admin] PATH_INFO: `/v3/projects/1e10bc2823424ea6acfc0fc82a3890dd/users/71db141bb2f14a0bbe9a750427e54442/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:31.925988 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79cadb-4faa-46d0-a35e-3e05db5b8131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:31.929199 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79cadb-4faa-46d0-a35e-3e05db5b8131 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:31.929659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79cadb-4faa-46d0-a35e-3e05db5b8131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:31.930438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79cadb-4faa-46d0-a35e-3e05db5b8131 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:31.932507 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79cadb-4faa-46d0-a35e-3e05db5b8131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:31.936624 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79cadb-4faa-46d0-a35e-3e05db5b8131 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:31.953381 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f458f1-d2e1-454b-90fd-fee3b4ff2fbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e10bc2823424ea6acfc0fc82a3890dd, user_id=71db141bb2f14a0bbe9a750427e54442, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:31.960481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f458f1-d2e1-454b-90fd-fee3b4ff2fbe admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:32.129783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 851/1722] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:31 2022] PUT /identity/v3/projects/1e10bc2823424ea6acfc0fc82a3890dd/users/71db141bb2f14a0bbe9a750427e54442/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:32.136481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8069d1-0156-4fd6-975b-f0f7e44a878f None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:32.136775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8069d1-0156-4fd6-975b-f0f7e44a878f None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:32.138975 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 872/1723] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:31 2022] POST /identity/v3/projects => generated 381 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:32.140254 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8069d1-0156-4fd6-975b-f0f7e44a878f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:32.143205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e647c28-df82-48e0-a0e1-e55f580b81cd None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:32.143205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e647c28-df82-48e0-a0e1-e55f580b81cd None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:32.143509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e647c28-df82-48e0-a0e1-e55f580b81cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:32.220937 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-8c8069d1-0156-4fd6-975b-f0f7e44a878f None None] MFA Rules not processed for user `91c691b6a7724d708bfc7424908516c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:32.223772 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8c8069d1-0156-4fd6-975b-f0f7e44a878f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:32.225571 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-5e647c28-df82-48e0-a0e1-e55f580b81cd None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:32.228288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5e647c28-df82-48e0-a0e1-e55f580b81cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:32.322889 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 852/1724] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:32.330723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8069d1-0156-4fd6-975b-f0f7e44a878f None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:32.389164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 873/1725] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:32.401127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e647c28-df82-48e0-a0e1-e55f580b81cd None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:32.415168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5e647c28-df82-48e0-a0e1-e55f580b81cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:32.537872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e647c28-df82-48e0-a0e1-e55f580b81cd None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:32.539458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8069d1-0156-4fd6-975b-f0f7e44a878f None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:32.541466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a74ec-22a5-4cc5-bde2-79eed215af3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:32.542103 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a74ec-22a5-4cc5-bde2-79eed215af3e service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:32.542406 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a74ec-22a5-4cc5-bde2-79eed215af3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:32.542748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a74ec-22a5-4cc5-bde2-79eed215af3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:32.551630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe661a13-0b96-4117-9d24-b3e6b9197bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:32.552458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe661a13-0b96-4117-9d24-b3e6b9197bbd admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:32.553239 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe661a13-0b96-4117-9d24-b3e6b9197bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:32.553596 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe661a13-0b96-4117-9d24-b3e6b9197bbd admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:32.554765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a74ec-22a5-4cc5-bde2-79eed215af3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:32.555513 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe661a13-0b96-4117-9d24-b3e6b9197bbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:32.556793 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a74ec-22a5-4cc5-bde2-79eed215af3e service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:32.560156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe661a13-0b96-4117-9d24-b3e6b9197bbd admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:32.570731 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 874/1726] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:32.753239 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 853/1727] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:32 2022] POST /identity/v3/users => generated 291 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:32.756223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a74ec-22a5-4cc5-bde2-79eed215af3e service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:32.770064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a74ec-22a5-4cc5-bde2-79eed215af3e service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:32.790577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18941c-e306-4b21-81b9-42ef78cec32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:32.791469 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18941c-e306-4b21-81b9-42ef78cec32f admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:32.791754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18941c-e306-4b21-81b9-42ef78cec32f admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:32.792031 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18941c-e306-4b21-81b9-42ef78cec32f admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:32.792741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18941c-e306-4b21-81b9-42ef78cec32f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:32.794860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18941c-e306-4b21-81b9-42ef78cec32f admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:32.839142 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 875/1728] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:32.847680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe661a13-0b96-4117-9d24-b3e6b9197bbd admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:32.864656 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe661a13-0b96-4117-9d24-b3e6b9197bbd admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:32.876062 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872d05d-7588-414c-90f0-8eaf6ba62091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:32.878093 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872d05d-7588-414c-90f0-8eaf6ba62091 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:32.878983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872d05d-7588-414c-90f0-8eaf6ba62091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:32.879387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872d05d-7588-414c-90f0-8eaf6ba62091 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:32.882849 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e872d05d-7588-414c-90f0-8eaf6ba62091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:32.887437 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e872d05d-7588-414c-90f0-8eaf6ba62091 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:32.917956 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 854/1729] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:32.922694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18941c-e306-4b21-81b9-42ef78cec32f admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:32.952609 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18941c-e306-4b21-81b9-42ef78cec32f admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:32.968451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ea96e-6176-4a39-a7ee-ff2a122df8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:32.973049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ea96e-6176-4a39-a7ee-ff2a122df8f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:32.973049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ea96e-6176-4a39-a7ee-ff2a122df8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:32.973049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ea96e-6176-4a39-a7ee-ff2a122df8f1 admin admin] PATH_INFO: `/v3/projects/421f588f4cf54879930ec577acf54db9/users/b0dd25348e2c434f80c5c6712827b527/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:33.019636 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ea96e-6176-4a39-a7ee-ff2a122df8f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=421f588f4cf54879930ec577acf54db9, user_id=b0dd25348e2c434f80c5c6712827b527, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:33.036947 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ea96e-6176-4a39-a7ee-ff2a122df8f1 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:33.129834 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:33.170064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 876/1730] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:32 2022] PUT /identity/v3/projects/421f588f4cf54879930ec577acf54db9/users/b0dd25348e2c434f80c5c6712827b527/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:33.172401 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc6450-ea0c-4071-99a4-b915f2a23733 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:33.172790 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc6450-ea0c-4071-99a4-b915f2a23733 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:33.173145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc6450-ea0c-4071-99a4-b915f2a23733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:33.194330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:33.236294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-39fc6450-ea0c-4071-99a4-b915f2a23733 None None] MFA Rules not processed for user `b0dd25348e2c434f80c5c6712827b527`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:33.263409 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-39fc6450-ea0c-4071-99a4-b915f2a23733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:33.436967 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 855/1731] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:33.442219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ea96e-6176-4a39-a7ee-ff2a122df8f1 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:33.659665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ea96e-6176-4a39-a7ee-ff2a122df8f1 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:33.665759 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212d186-33dd-4199-89de-9774db42738d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:33.670626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212d186-33dd-4199-89de-9774db42738d admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:33.670626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212d186-33dd-4199-89de-9774db42738d admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:33.670626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212d186-33dd-4199-89de-9774db42738d admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:33.670626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212d186-33dd-4199-89de-9774db42738d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:33.671746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212d186-33dd-4199-89de-9774db42738d admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:33.872549 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 877/1732] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:33 2022] POST /identity/v3/projects => generated 381 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:33.876202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc6450-ea0c-4071-99a4-b915f2a23733 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:33.893237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc6450-ea0c-4071-99a4-b915f2a23733 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:34.058109 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07260c63-4d0c-4442-a3c9-8f243742ed0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:34.059479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07260c63-4d0c-4442-a3c9-8f243742ed0c admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:34.059945 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07260c63-4d0c-4442-a3c9-8f243742ed0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:34.060363 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07260c63-4d0c-4442-a3c9-8f243742ed0c admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:34.061163 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07260c63-4d0c-4442-a3c9-8f243742ed0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:34.069377 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07260c63-4d0c-4442-a3c9-8f243742ed0c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:34.250940 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 856/1733] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:33 2022] POST /identity/v3/users => generated 297 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:34.251106 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212d186-33dd-4199-89de-9774db42738d admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:34.269329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212d186-33dd-4199-89de-9774db42738d admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:34.279424 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80870255-b6a7-4330-b280-4d50d455008e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:34.280477 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80870255-b6a7-4330-b280-4d50d455008e admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:34.280751 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80870255-b6a7-4330-b280-4d50d455008e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:34.281019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80870255-b6a7-4330-b280-4d50d455008e admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:34.281681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80870255-b6a7-4330-b280-4d50d455008e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:34.287387 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80870255-b6a7-4330-b280-4d50d455008e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:34.311964 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 878/1734] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:34.313625 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07260c63-4d0c-4442-a3c9-8f243742ed0c admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:34.335412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07260c63-4d0c-4442-a3c9-8f243742ed0c admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:34.358456 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f98466-eb6a-4963-a0b5-1c207c46aef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:34.360192 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f98466-eb6a-4963-a0b5-1c207c46aef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:34.360939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f98466-eb6a-4963-a0b5-1c207c46aef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:34.363655 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f98466-eb6a-4963-a0b5-1c207c46aef0 admin admin] PATH_INFO: `/v3/projects/1dbab19e2797462ba71dcb40202eac97/users/c34b0d812bda46afb0d52702894ee3f7/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:34.371761 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:34.410525 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f98466-eb6a-4963-a0b5-1c207c46aef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dbab19e2797462ba71dcb40202eac97, user_id=c34b0d812bda46afb0d52702894ee3f7, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:34.419833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f98466-eb6a-4963-a0b5-1c207c46aef0 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:34.553135 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 857/1735] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:34 2022] PUT /identity/v3/projects/1dbab19e2797462ba71dcb40202eac97/users/c34b0d812bda46afb0d52702894ee3f7/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:34.557267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80870255-b6a7-4330-b280-4d50d455008e admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:34.568250 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80870255-b6a7-4330-b280-4d50d455008e admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:34.648115 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12240ba2-4593-4a6f-86a1-af12c125cfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:34.648729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12240ba2-4593-4a6f-86a1-af12c125cfec admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:34.649169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12240ba2-4593-4a6f-86a1-af12c125cfec admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:34.649579 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12240ba2-4593-4a6f-86a1-af12c125cfec admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:34.652832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12240ba2-4593-4a6f-86a1-af12c125cfec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:34.652832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12240ba2-4593-4a6f-86a1-af12c125cfec admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:34.663999 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 879/1736] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:34.668945 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f98466-eb6a-4963-a0b5-1c207c46aef0 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:34.752704 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f98466-eb6a-4963-a0b5-1c207c46aef0 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:34.780451 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970842a-a694-45fa-bae9-fe028bc9e772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:34.781554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970842a-a694-45fa-bae9-fe028bc9e772 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:34.783003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970842a-a694-45fa-bae9-fe028bc9e772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:34.783865 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970842a-a694-45fa-bae9-fe028bc9e772 admin admin] PATH_INFO: `/v3/projects/1dbab19e2797462ba71dcb40202eac97/users/c34b0d812bda46afb0d52702894ee3f7/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:34.786039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970842a-a694-45fa-bae9-fe028bc9e772 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dbab19e2797462ba71dcb40202eac97, user_id=c34b0d812bda46afb0d52702894ee3f7, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:34.789557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970842a-a694-45fa-bae9-fe028bc9e772 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:34.803229 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 858/1737] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:34 2022] PUT /identity/v3/projects/1dbab19e2797462ba71dcb40202eac97/users/c34b0d812bda46afb0d52702894ee3f7/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:34.813353 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72db2c-29f8-4d9b-97ca-96144ebf04d5 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:34.813643 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72db2c-29f8-4d9b-97ca-96144ebf04d5 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:34.813923 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72db2c-29f8-4d9b-97ca-96144ebf04d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:34.841279 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f890668-9afa-48d2-aa65-a80bf184ecf2 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:34.841968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f890668-9afa-48d2-aa65-a80bf184ecf2 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:34.842390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f890668-9afa-48d2-aa65-a80bf184ecf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:34.872718 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-cb72db2c-29f8-4d9b-97ca-96144ebf04d5 None None] MFA Rules not processed for user `c34b0d812bda46afb0d52702894ee3f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:34.873950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-cb72db2c-29f8-4d9b-97ca-96144ebf04d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:34.891468 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-3f890668-9afa-48d2-aa65-a80bf184ecf2 None None] MFA Rules not processed for user `71db141bb2f14a0bbe9a750427e54442`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:34.896051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-3f890668-9afa-48d2-aa65-a80bf184ecf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:35.082072 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 859/1738] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:34:35.086667 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f890668-9afa-48d2-aa65-a80bf184ecf2 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:35.088460 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 880/1739] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:35.105413 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72db2c-29f8-4d9b-97ca-96144ebf04d5 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:35.106847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-cb72db2c-29f8-4d9b-97ca-96144ebf04d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:35.170866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f890668-9afa-48d2-aa65-a80bf184ecf2 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:35.177197 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08eff3-1da7-4245-96de-60f1817f1d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:35.177815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08eff3-1da7-4245-96de-60f1817f1d37 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:35.178274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08eff3-1da7-4245-96de-60f1817f1d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:35.178869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08eff3-1da7-4245-96de-60f1817f1d37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:35.179652 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08eff3-1da7-4245-96de-60f1817f1d37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:35.181880 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08eff3-1da7-4245-96de-60f1817f1d37 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:35.192402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72db2c-29f8-4d9b-97ca-96144ebf04d5 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:35.196350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db605c68-3994-4e48-8cf2-01cf8aeff4b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:35.197180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db605c68-3994-4e48-8cf2-01cf8aeff4b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:35.197652 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db605c68-3994-4e48-8cf2-01cf8aeff4b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:35.198066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db605c68-3994-4e48-8cf2-01cf8aeff4b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:35.209668 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db605c68-3994-4e48-8cf2-01cf8aeff4b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:35.212091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db605c68-3994-4e48-8cf2-01cf8aeff4b2 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:35.228254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 881/1740] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:35.335026 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 860/1741] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:35 2022] POST /identity/v3/projects => generated 373 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:35.340067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db605c68-3994-4e48-8cf2-01cf8aeff4b2 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:35.371401 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db605c68-3994-4e48-8cf2-01cf8aeff4b2 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:35.514957 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb4f71-e771-4bb5-a2fe-96460d32fdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:35.516358 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb4f71-e771-4bb5-a2fe-96460d32fdde admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:35.516358 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb4f71-e771-4bb5-a2fe-96460d32fdde admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:35.516569 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb4f71-e771-4bb5-a2fe-96460d32fdde admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:35.517204 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb4f71-e771-4bb5-a2fe-96460d32fdde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:35.538354 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb4f71-e771-4bb5-a2fe-96460d32fdde admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:35.700456 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 882/1742] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:35 2022] POST /identity/v3/users => generated 293 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:35.701402 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08eff3-1da7-4245-96de-60f1817f1d37 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:35.715212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08eff3-1da7-4245-96de-60f1817f1d37 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:35.722997 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce61bc-68d7-4ff4-948f-9abd887687e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:35.724004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce61bc-68d7-4ff4-948f-9abd887687e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:35.724367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce61bc-68d7-4ff4-948f-9abd887687e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:35.724713 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce61bc-68d7-4ff4-948f-9abd887687e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:35.725655 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce61bc-68d7-4ff4-948f-9abd887687e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:35.727432 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce61bc-68d7-4ff4-948f-9abd887687e4 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:35.741436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 861/1743] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:35.746671 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb4f71-e771-4bb5-a2fe-96460d32fdde admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:35.767480 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb4f71-e771-4bb5-a2fe-96460d32fdde admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:35.776893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44d36e-264b-4633-8c42-df8871782e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:35.777868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44d36e-264b-4633-8c42-df8871782e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:35.778590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44d36e-264b-4633-8c42-df8871782e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:35.778811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44d36e-264b-4633-8c42-df8871782e2a admin admin] PATH_INFO: `/v3/projects/d4f8d27ef3d849f2ac867eea735c557e/users/c3bf549dcf834ce1922b1ab29f2d9fba/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:35.803511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44d36e-264b-4633-8c42-df8871782e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4f8d27ef3d849f2ac867eea735c557e, user_id=c3bf549dcf834ce1922b1ab29f2d9fba, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:35.808703 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44d36e-264b-4633-8c42-df8871782e2a admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:35.958380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 883/1744] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:35 2022] PUT /identity/v3/projects/d4f8d27ef3d849f2ac867eea735c557e/users/c3bf549dcf834ce1922b1ab29f2d9fba/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:35.962515 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce61bc-68d7-4ff4-948f-9abd887687e4 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:35.974997 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce61bc-68d7-4ff4-948f-9abd887687e4 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:36.172180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1190d4-78f4-4402-a88d-f57f7a87d4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:36.174608 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1190d4-78f4-4402-a88d-f57f7a87d4c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:36.175112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1190d4-78f4-4402-a88d-f57f7a87d4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:36.175939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1190d4-78f4-4402-a88d-f57f7a87d4c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:36.183366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1190d4-78f4-4402-a88d-f57f7a87d4c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:36.183366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1190d4-78f4-4402-a88d-f57f7a87d4c7 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:36.188932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 862/1745] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:36.193184 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44d36e-264b-4633-8c42-df8871782e2a admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:36.222883 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44d36e-264b-4633-8c42-df8871782e2a admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:36.234727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2976a86-9cf2-42d0-90c3-5249dadf2e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:36.237763 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2976a86-9cf2-42d0-90c3-5249dadf2e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:36.253261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2976a86-9cf2-42d0-90c3-5249dadf2e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:36.253662 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2976a86-9cf2-42d0-90c3-5249dadf2e30 admin admin] PATH_INFO: `/v3/projects/d4f8d27ef3d849f2ac867eea735c557e/users/c3bf549dcf834ce1922b1ab29f2d9fba/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:36.258662 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2976a86-9cf2-42d0-90c3-5249dadf2e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4f8d27ef3d849f2ac867eea735c557e, user_id=c3bf549dcf834ce1922b1ab29f2d9fba, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:36.263213 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2976a86-9cf2-42d0-90c3-5249dadf2e30 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:36.298090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 884/1746] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:36 2022] PUT /identity/v3/projects/d4f8d27ef3d849f2ac867eea735c557e/users/c3bf549dcf834ce1922b1ab29f2d9fba/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:34:36.301654 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390797f-f8a3-4d21-90a8-e25f8c556ed2 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:36.304186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390797f-f8a3-4d21-90a8-e25f8c556ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:36.304814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390797f-f8a3-4d21-90a8-e25f8c556ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:36.407264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-c390797f-f8a3-4d21-90a8-e25f8c556ed2 None None] MFA Rules not processed for user `86300433c724483994f59709a3f015db`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:36.409026 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c390797f-f8a3-4d21-90a8-e25f8c556ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:36.642064 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 863/1747] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 5 14:34:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:36.881488 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:37.087854 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2976a86-9cf2-42d0-90c3-5249dadf2e30 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:37.127510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390797f-f8a3-4d21-90a8-e25f8c556ed2 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:37.129381 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c390797f-f8a3-4d21-90a8-e25f8c556ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:37.140516 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:37.347572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2976a86-9cf2-42d0-90c3-5249dadf2e30 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:37.349961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390797f-f8a3-4d21-90a8-e25f8c556ed2 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:37.360098 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755b904-4dc9-4f59-937b-19db769c5bf1 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34b0d812bda46afb0d52702894ee3f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dbab19e2797462ba71dcb40202eac97', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:37.362296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755b904-4dc9-4f59-937b-19db769c5bf1 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:37.363173 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755b904-4dc9-4f59-937b-19db769c5bf1 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:37.365088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755b904-4dc9-4f59-937b-19db769c5bf1 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:37.366076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f7b3a-7dc8-459a-bf58-1954afa95084 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a85809d14cb449d848f22a3f39bf641', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:37.366955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f7b3a-7dc8-459a-bf58-1954afa95084 service swift] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:37.367550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1755b904-4dc9-4f59-937b-19db769c5bf1 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:37.368081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f7b3a-7dc8-459a-bf58-1954afa95084 service swift] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:37.368533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f7b3a-7dc8-459a-bf58-1954afa95084 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:37.370823 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1755b904-4dc9-4f59-937b-19db769c5bf1 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:37.373101 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1755b904-4dc9-4f59-937b-19db769c5bf1 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:37.391847 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f7b3a-7dc8-459a-bf58-1954afa95084 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:37.395744 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f7b3a-7dc8-459a-bf58-1954afa95084 service swift] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:37.417059 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 864/1748] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 5 14:34:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 14:34:37.421003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 885/1749] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:34:37.421572 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f7b3a-7dc8-459a-bf58-1954afa95084 service swift] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:37.446150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f7b3a-7dc8-459a-bf58-1954afa95084 service swift] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:37.462359 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ed37c-7eff-48fb-8d86-87ee2724472e tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34b0d812bda46afb0d52702894ee3f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dbab19e2797462ba71dcb40202eac97', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:37.464268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ed37c-7eff-48fb-8d86-87ee2724472e tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:37.464783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ed37c-7eff-48fb-8d86-87ee2724472e tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:37.465496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ed37c-7eff-48fb-8d86-87ee2724472e tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:37.466325 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ed37c-7eff-48fb-8d86-87ee2724472e tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:37.468488 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ed37c-7eff-48fb-8d86-87ee2724472e tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:37.623079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 865/1750] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:37 2022] POST /identity/v3/projects => generated 353 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:37.626575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755b904-4dc9-4f59-937b-19db769c5bf1 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:37.641520 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755b904-4dc9-4f59-937b-19db769c5bf1 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:37.835156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf16f33-76b6-471e-a211-090e9299fae2 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34b0d812bda46afb0d52702894ee3f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dbab19e2797462ba71dcb40202eac97', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:37.835878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf16f33-76b6-471e-a211-090e9299fae2 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:37.836375 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf16f33-76b6-471e-a211-090e9299fae2 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:37.836815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf16f33-76b6-471e-a211-090e9299fae2 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:37.837540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf16f33-76b6-471e-a211-090e9299fae2 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:37.837862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf16f33-76b6-471e-a211-090e9299fae2 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:37.840316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf16f33-76b6-471e-a211-090e9299fae2 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:37.887806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 886/1751] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:34:37.997676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ed37c-7eff-48fb-8d86-87ee2724472e tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:38.013337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ed37c-7eff-48fb-8d86-87ee2724472e tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:38.027254 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797417c8-c948-48bc-a7c7-90c5f63ad986 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34b0d812bda46afb0d52702894ee3f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dbab19e2797462ba71dcb40202eac97', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:38.029914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797417c8-c948-48bc-a7c7-90c5f63ad986 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:38.030158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797417c8-c948-48bc-a7c7-90c5f63ad986 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:38.031713 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797417c8-c948-48bc-a7c7-90c5f63ad986 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] PATH_INFO: `/v3/projects/ece072ea0f0543fbbf0ba7331dcbdde5` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:38.033434 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797417c8-c948-48bc-a7c7-90c5f63ad986 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ece072ea0f0543fbbf0ba7331dcbdde5)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:38.035519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797417c8-c948-48bc-a7c7-90c5f63ad986 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:38.328054 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-797417c8-c948-48bc-a7c7-90c5f63ad986 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ece072ea0f0543fbbf0ba7331dcbdde5', 'request_id': 'req-797417c8-c948-48bc-a7c7-90c5f63ad986'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:38.339983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-797417c8-c948-48bc-a7c7-90c5f63ad986 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ece072ea0f0543fbbf0ba7331dcbdde5', 'request_id': 'req-797417c8-c948-48bc-a7c7-90c5f63ad986'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:38.451199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-797417c8-c948-48bc-a7c7-90c5f63ad986 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ece072ea0f0543fbbf0ba7331dcbdde5', 'request_id': 'req-797417c8-c948-48bc-a7c7-90c5f63ad986'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:38.469084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-797417c8-c948-48bc-a7c7-90c5f63ad986 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] The token cache is being invalidate because project ece072ea0f0543fbbf0ba7331dcbdde5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:34:38.472062 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-797417c8-c948-48bc-a7c7-90c5f63ad986 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:38.474520 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-797417c8-c948-48bc-a7c7-90c5f63ad986 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:38.480912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 866/1752] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:34:37 2022] DELETE /identity/v3/projects/ece072ea0f0543fbbf0ba7331dcbdde5 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:39.539889 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:39.689800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:40.173502 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf16f33-76b6-471e-a211-090e9299fae2 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:40.175911 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-fbf16f33-76b6-471e-a211-090e9299fae2 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:40.228169 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9bd1e-e52b-4814-8640-496c9382c297 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:40.228609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9bd1e-e52b-4814-8640-496c9382c297 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:40.228984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9bd1e-e52b-4814-8640-496c9382c297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:40.311372 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-4cc9bd1e-e52b-4814-8640-496c9382c297 None None] MFA Rules not processed for user `5c73ca2785334fc496b5227d3be655c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:40.311372 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-4cc9bd1e-e52b-4814-8640-496c9382c297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:40.418666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf16f33-76b6-471e-a211-090e9299fae2 tempest-VolumeQuotaClassesTest-1327624319 tempest-VolumeQuotaClassesTest-1327624319-project-admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:40.568894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 867/1753] 10.222.0.50 () {58 vars in 1015 bytes} [Mon Sep 5 14:34:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:40.648637 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7f6e8-271a-447a-a913-31417ab916b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:40.649282 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7f6e8-271a-447a-a913-31417ab916b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:40.649637 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7f6e8-271a-447a-a913-31417ab916b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:40.649896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7f6e8-271a-447a-a913-31417ab916b7 admin admin] PATH_INFO: `/v3/users/b0dd25348e2c434f80c5c6712827b527` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:40.651249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7f6e8-271a-447a-a913-31417ab916b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0dd25348e2c434f80c5c6712827b527)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:40.653205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7f6e8-271a-447a-a913-31417ab916b7 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:40.696400 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9bd1e-e52b-4814-8640-496c9382c297 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:40.711026 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-4cc9bd1e-e52b-4814-8640-496c9382c297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:40.721187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9bd1e-e52b-4814-8640-496c9382c297 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:40.731303 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5f63e-4b3b-4e77-85ad-6e4db00fa981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:40.732249 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5f63e-4b3b-4e77-85ad-6e4db00fa981 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:40.732809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5f63e-4b3b-4e77-85ad-6e4db00fa981 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:40.733242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5f63e-4b3b-4e77-85ad-6e4db00fa981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:40.763800 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5f63e-4b3b-4e77-85ad-6e4db00fa981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:40.765152 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5f63e-4b3b-4e77-85ad-6e4db00fa981 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:40.783681 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 868/1754] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:34:40.784174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5f63e-4b3b-4e77-85ad-6e4db00fa981 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:40.792562 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ded5f63e-4b3b-4e77-85ad-6e4db00fa981 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:40.824864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5f63e-4b3b-4e77-85ad-6e4db00fa981 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:40.837529 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a271a3-daed-45dc-8568-78af5254b494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:40.838422 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a271a3-daed-45dc-8568-78af5254b494 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:40.838707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a271a3-daed-45dc-8568-78af5254b494 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:40.839013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a271a3-daed-45dc-8568-78af5254b494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:40.857842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a271a3-daed-45dc-8568-78af5254b494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:40.859663 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a271a3-daed-45dc-8568-78af5254b494 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:40.889794 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 869/1755] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:40.971604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-2db7f6e8-271a-447a-a913-31417ab916b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0dd25348e2c434f80c5c6712827b527'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:41.071975 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-2db7f6e8-271a-447a-a913-31417ab916b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0dd25348e2c434f80c5c6712827b527'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:41.102572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-2db7f6e8-271a-447a-a913-31417ab916b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0dd25348e2c434f80c5c6712827b527'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:41.103357 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-2db7f6e8-271a-447a-a913-31417ab916b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0dd25348e2c434f80c5c6712827b527'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:41.116476 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-2db7f6e8-271a-447a-a913-31417ab916b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0dd25348e2c434f80c5c6712827b527'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:41.141843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078a3e4-56cd-4da9-85d4-f178759f4ae6 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:41.141843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078a3e4-56cd-4da9-85d4-f178759f4ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:41.141998 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 887/1756] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:34:40 2022] DELETE /identity/v3/users/b0dd25348e2c434f80c5c6712827b527 => generated 0 bytes in 964 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:41.142099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078a3e4-56cd-4da9-85d4-f178759f4ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:41.193837 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-d078a3e4-56cd-4da9-85d4-f178759f4ae6 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:41.195423 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d078a3e4-56cd-4da9-85d4-f178759f4ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:41.364978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 870/1757] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:34:41.396308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7f6e8-271a-447a-a913-31417ab916b7 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:41.404044 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-2db7f6e8-271a-447a-a913-31417ab916b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:41.512392 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:41.521552 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7f6e8-271a-447a-a913-31417ab916b7 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:41.528208 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af95f4b-b95d-4b8e-9153-3b5f14e9ad88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:41.529194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af95f4b-b95d-4b8e-9153-3b5f14e9ad88 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:41.529622 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af95f4b-b95d-4b8e-9153-3b5f14e9ad88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:41.530086 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af95f4b-b95d-4b8e-9153-3b5f14e9ad88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:41.543794 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af95f4b-b95d-4b8e-9153-3b5f14e9ad88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:41.546216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af95f4b-b95d-4b8e-9153-3b5f14e9ad88 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:41.558674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 888/1758] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:41.603696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:42.691928 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:43.032552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bcff9-1716-4832-b7a9-51b9e670f24e None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:43.032929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bcff9-1716-4832-b7a9-51b9e670f24e None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:43.033801 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bcff9-1716-4832-b7a9-51b9e670f24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:43.099745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-f45bcff9-1716-4832-b7a9-51b9e670f24e None None] MFA Rules not processed for user `c3bf549dcf834ce1922b1ab29f2d9fba`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:43.101082 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f45bcff9-1716-4832-b7a9-51b9e670f24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:43.271381 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 871/1759] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:43.280550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af95f4b-b95d-4b8e-9153-3b5f14e9ad88 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:43.282730 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0af95f4b-b95d-4b8e-9153-3b5f14e9ad88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:43.507903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af95f4b-b95d-4b8e-9153-3b5f14e9ad88 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:43.530049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766b16f-4840-40f7-8c04-81705eb33284 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:43.530049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766b16f-4840-40f7-8c04-81705eb33284 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:43.530049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766b16f-4840-40f7-8c04-81705eb33284 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:43.530049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766b16f-4840-40f7-8c04-81705eb33284 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:43.530717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-2766b16f-4840-40f7-8c04-81705eb33284 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:43.617018 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766b16f-4840-40f7-8c04-81705eb33284 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:43.619298 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766b16f-4840-40f7-8c04-81705eb33284 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:43.628767 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bcff9-1716-4832-b7a9-51b9e670f24e None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:43.638304 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f45bcff9-1716-4832-b7a9-51b9e670f24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:43.676649 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 889/1760] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:43.678716 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bcff9-1716-4832-b7a9-51b9e670f24e None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:43.695575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56506eca-05d2-4b29-9f0f-d2f1db6008f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:43.698124 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56506eca-05d2-4b29-9f0f-d2f1db6008f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:43.699464 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56506eca-05d2-4b29-9f0f-d2f1db6008f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:43.702258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56506eca-05d2-4b29-9f0f-d2f1db6008f7 admin admin] PATH_INFO: `/v3/projects/421f588f4cf54879930ec577acf54db9` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:43.703814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56506eca-05d2-4b29-9f0f-d2f1db6008f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=421f588f4cf54879930ec577acf54db9)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:43.709587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56506eca-05d2-4b29-9f0f-d2f1db6008f7 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:43.887699 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766b16f-4840-40f7-8c04-81705eb33284 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:43.887699 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-2766b16f-4840-40f7-8c04-81705eb33284 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:43.925549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-56506eca-05d2-4b29-9f0f-d2f1db6008f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '421f588f4cf54879930ec577acf54db9', 'request_id': 'req-56506eca-05d2-4b29-9f0f-d2f1db6008f7'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:43.948657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766b16f-4840-40f7-8c04-81705eb33284 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:43.952655 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-56506eca-05d2-4b29-9f0f-d2f1db6008f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '421f588f4cf54879930ec577acf54db9', 'request_id': 'req-56506eca-05d2-4b29-9f0f-d2f1db6008f7'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:44.037646 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:44.061311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-56506eca-05d2-4b29-9f0f-d2f1db6008f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '421f588f4cf54879930ec577acf54db9', 'request_id': 'req-56506eca-05d2-4b29-9f0f-d2f1db6008f7'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:44.103679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-56506eca-05d2-4b29-9f0f-d2f1db6008f7 admin admin] The token cache is being invalidate because project 421f588f4cf54879930ec577acf54db9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:34:44.104843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-56506eca-05d2-4b29-9f0f-d2f1db6008f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:44.109917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-56506eca-05d2-4b29-9f0f-d2f1db6008f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:44.114404 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 872/1761] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:34:43 2022] DELETE /identity/v3/projects/421f588f4cf54879930ec577acf54db9 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:44.116078 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4502b1-be9d-4753-893a-a23eb55a8c69 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:44.116528 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4502b1-be9d-4753-893a-a23eb55a8c69 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:44.117452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4502b1-be9d-4753-893a-a23eb55a8c69 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:44.128346 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 873/1762] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 5 14:34:44 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:34:44.131938 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bd2ce-7846-46e3-b6a0-d20fde509f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:44.132735 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bd2ce-7846-46e3-b6a0-d20fde509f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:44.133021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bd2ce-7846-46e3-b6a0-d20fde509f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:44.133300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bd2ce-7846-46e3-b6a0-d20fde509f42 admin admin] PATH_INFO: `/v3/users/624b9ced974f476e8ef61dcb89aa631d` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:44.133754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4502b1-be9d-4753-893a-a23eb55a8c69 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:44.138003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bd2ce-7846-46e3-b6a0-d20fde509f42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=624b9ced974f476e8ef61dcb89aa631d)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:44.139556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-6e4502b1-be9d-4753-893a-a23eb55a8c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:44.142427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bd2ce-7846-46e3-b6a0-d20fde509f42 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:44.197091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4502b1-be9d-4753-893a-a23eb55a8c69 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:44.204779 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad4ec2f-f49d-4258-92fb-e695deb7d34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:44.205769 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4ec2f-f49d-4258-92fb-e695deb7d34a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:44.206201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4ec2f-f49d-4258-92fb-e695deb7d34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:44.206766 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4ec2f-f49d-4258-92fb-e695deb7d34a admin admin] PATH_INFO: `/v3/users/c34b0d812bda46afb0d52702894ee3f7` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:44.208824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad4ec2f-f49d-4258-92fb-e695deb7d34a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c34b0d812bda46afb0d52702894ee3f7)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:44.211130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad4ec2f-f49d-4258-92fb-e695deb7d34a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:44.495993 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-eb8bd2ce-7846-46e3-b6a0-d20fde509f42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '624b9ced974f476e8ef61dcb89aa631d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:44.512900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-5ad4ec2f-f49d-4258-92fb-e695deb7d34a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c34b0d812bda46afb0d52702894ee3f7'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:44.637203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-eb8bd2ce-7846-46e3-b6a0-d20fde509f42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '624b9ced974f476e8ef61dcb89aa631d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:44.701333 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-eb8bd2ce-7846-46e3-b6a0-d20fde509f42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '624b9ced974f476e8ef61dcb89aa631d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:44.704191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-eb8bd2ce-7846-46e3-b6a0-d20fde509f42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '624b9ced974f476e8ef61dcb89aa631d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:44.726563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-5ad4ec2f-f49d-4258-92fb-e695deb7d34a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c34b0d812bda46afb0d52702894ee3f7'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:44.728319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-5ad4ec2f-f49d-4258-92fb-e695deb7d34a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c34b0d812bda46afb0d52702894ee3f7'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:44.769069 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-eb8bd2ce-7846-46e3-b6a0-d20fde509f42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '624b9ced974f476e8ef61dcb89aa631d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:44.779867 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-5ad4ec2f-f49d-4258-92fb-e695deb7d34a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c34b0d812bda46afb0d52702894ee3f7'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:44.810914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 890/1763] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:34:43 2022] DELETE /identity/v3/users/624b9ced974f476e8ef61dcb89aa631d => generated 0 bytes in 923 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:34:44.810914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b7a89-09f3-4fac-8257-e8f576f88145 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:44.810914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b7a89-09f3-4fac-8257-e8f576f88145 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:44.810914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b7a89-09f3-4fac-8257-e8f576f88145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:44.825683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-5ad4ec2f-f49d-4258-92fb-e695deb7d34a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c34b0d812bda46afb0d52702894ee3f7'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:44.840526 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 874/1764] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:34:44 2022] DELETE /identity/v3/users/c34b0d812bda46afb0d52702894ee3f7 => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:44.842517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d83ed5-f09f-4694-b75a-41744845dbf8 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:44.842963 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d83ed5-f09f-4694-b75a-41744845dbf8 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:44.843646 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d83ed5-f09f-4694-b75a-41744845dbf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:44.886491 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-d81b7a89-09f3-4fac-8257-e8f576f88145 None None] MFA Rules not processed for user `709b9ae82dcd47408fc9a6d54a54428a`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:44.898019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d81b7a89-09f3-4fac-8257-e8f576f88145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:44.943156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-99d83ed5-f09f-4694-b75a-41744845dbf8 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:44.943156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-99d83ed5-f09f-4694-b75a-41744845dbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:45.073904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 891/1765] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 5 14:34:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:45.073904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fffda-0808-42be-9955-124ce7ea89e5 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:45.073904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fffda-0808-42be-9955-124ce7ea89e5 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:45.073904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fffda-0808-42be-9955-124ce7ea89e5 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:45.078333 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 892/1766] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 5 14:34:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:34:45.136190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 875/1767] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:45.246402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a39ab2-e296-44de-bd14-7af043651ebe None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:45.246402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a39ab2-e296-44de-bd14-7af043651ebe None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:45.246402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a39ab2-e296-44de-bd14-7af043651ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:45.285249 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d83ed5-f09f-4694-b75a-41744845dbf8 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:45.287115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-99d83ed5-f09f-4694-b75a-41744845dbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:45.291608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-65a39ab2-e296-44de-bd14-7af043651ebe None None] MFA Rules not processed for user `709b9ae82dcd47408fc9a6d54a54428a`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:45.295080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-65a39ab2-e296-44de-bd14-7af043651ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:45.322465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 893/1768] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 5 14:34:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:45.364704 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a39ab2-e296-44de-bd14-7af043651ebe None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:45.366401 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-65a39ab2-e296-44de-bd14-7af043651ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:45.393586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a39ab2-e296-44de-bd14-7af043651ebe None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:45.415036 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba66b27f-f6d0-4226-a269-7e9e9d44834c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:45.416011 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66b27f-f6d0-4226-a269-7e9e9d44834c service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:45.416616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66b27f-f6d0-4226-a269-7e9e9d44834c service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:45.417131 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66b27f-f6d0-4226-a269-7e9e9d44834c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:45.429036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d83ed5-f09f-4694-b75a-41744845dbf8 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:45.434633 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a4469-8d92-4357-96d9-88a6effdabb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:45.435359 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a4469-8d92-4357-96d9-88a6effdabb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:45.435994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a4469-8d92-4357-96d9-88a6effdabb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:45.436278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a4469-8d92-4357-96d9-88a6effdabb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:45.447403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba66b27f-f6d0-4226-a269-7e9e9d44834c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:45.447403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba66b27f-f6d0-4226-a269-7e9e9d44834c service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:45.448165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a4469-8d92-4357-96d9-88a6effdabb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:45.449964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a4469-8d92-4357-96d9-88a6effdabb4 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:45.462472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 876/1769] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:45.481889 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 894/1770] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:45.973474 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a4469-8d92-4357-96d9-88a6effdabb4 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:45.976006 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-833a4469-8d92-4357-96d9-88a6effdabb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:46.004874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a4469-8d92-4357-96d9-88a6effdabb4 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:46.012229 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a7ca5-26d4-41b0-af36-41da08ca4a08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:46.013030 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a7ca5-26d4-41b0-af36-41da08ca4a08 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:46.013360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a7ca5-26d4-41b0-af36-41da08ca4a08 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:46.013646 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a7ca5-26d4-41b0-af36-41da08ca4a08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:46.034864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a7ca5-26d4-41b0-af36-41da08ca4a08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:46.036810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a7ca5-26d4-41b0-af36-41da08ca4a08 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:46.058439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 877/1771] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:46.126636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:46.305793 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:46.991996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba66b27f-f6d0-4226-a269-7e9e9d44834c service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:47.006995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ba66b27f-f6d0-4226-a269-7e9e9d44834c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:47.163442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba66b27f-f6d0-4226-a269-7e9e9d44834c service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:47.168430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b5d6b-920e-4991-b1c7-86b7acfa4831 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:47.169065 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b5d6b-920e-4991-b1c7-86b7acfa4831 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:47.169343 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b5d6b-920e-4991-b1c7-86b7acfa4831 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:47.169858 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b5d6b-920e-4991-b1c7-86b7acfa4831 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:47.171819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-398b5d6b-920e-4991-b1c7-86b7acfa4831 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:47.201533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a7ca5-26d4-41b0-af36-41da08ca4a08 service nova] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:47.206516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-261a7ca5-26d4-41b0-af36-41da08ca4a08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:47.260508 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a7ca5-26d4-41b0-af36-41da08ca4a08 service nova] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:47.284745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ccbb7-f938-4d8c-ac5d-cb88ede4a24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:47.288101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ccbb7-f938-4d8c-ac5d-cb88ede4a24a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:47.290298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ccbb7-f938-4d8c-ac5d-cb88ede4a24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:47.291340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ccbb7-f938-4d8c-ac5d-cb88ede4a24a admin admin] PATH_INFO: `/v3/projects/516fa688c57b4dbda180bb54018846d6` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:47.293450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ccbb7-f938-4d8c-ac5d-cb88ede4a24a admin admin] RBAC: Authorizing `identity:delete_project(project_id=516fa688c57b4dbda180bb54018846d6)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:47.296937 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ccbb7-f938-4d8c-ac5d-cb88ede4a24a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:47.412447 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b5d6b-920e-4991-b1c7-86b7acfa4831 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:47.414267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b5d6b-920e-4991-b1c7-86b7acfa4831 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:47.429068 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 895/1772] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:47.434806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b5d6b-920e-4991-b1c7-86b7acfa4831 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:47.434806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-398b5d6b-920e-4991-b1c7-86b7acfa4831 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:47.453432 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b5d6b-920e-4991-b1c7-86b7acfa4831 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:47.462764 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc92a21-e52f-4d7e-a079-95ee5a53b087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:47.464389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc92a21-e52f-4d7e-a079-95ee5a53b087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:47.465445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc92a21-e52f-4d7e-a079-95ee5a53b087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:47.465848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc92a21-e52f-4d7e-a079-95ee5a53b087 admin admin] PATH_INFO: `/v3/projects/1dbab19e2797462ba71dcb40202eac97` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:47.467787 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc92a21-e52f-4d7e-a079-95ee5a53b087 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dbab19e2797462ba71dcb40202eac97)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:47.479393 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc92a21-e52f-4d7e-a079-95ee5a53b087 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:47.520483 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2f3ccbb7-f938-4d8c-ac5d-cb88ede4a24a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '516fa688c57b4dbda180bb54018846d6', 'request_id': 'req-2f3ccbb7-f938-4d8c-ac5d-cb88ede4a24a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:47.534679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2f3ccbb7-f938-4d8c-ac5d-cb88ede4a24a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '516fa688c57b4dbda180bb54018846d6', 'request_id': 'req-2f3ccbb7-f938-4d8c-ac5d-cb88ede4a24a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:47.633757 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2f3ccbb7-f938-4d8c-ac5d-cb88ede4a24a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '516fa688c57b4dbda180bb54018846d6', 'request_id': 'req-2f3ccbb7-f938-4d8c-ac5d-cb88ede4a24a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:47.669030 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2f3ccbb7-f938-4d8c-ac5d-cb88ede4a24a admin admin] The token cache is being invalidate because project 516fa688c57b4dbda180bb54018846d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:34:47.669322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2f3ccbb7-f938-4d8c-ac5d-cb88ede4a24a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:47.672287 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2f3ccbb7-f938-4d8c-ac5d-cb88ede4a24a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:47.678567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 878/1773] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:34:47 2022] DELETE /identity/v3/projects/516fa688c57b4dbda180bb54018846d6 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:47.684398 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52981288-ac3e-44dc-aaab-655b61c80bea None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:47.684564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52981288-ac3e-44dc-aaab-655b61c80bea None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:47.684766 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52981288-ac3e-44dc-aaab-655b61c80bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:47.703273 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-cdc92a21-e52f-4d7e-a079-95ee5a53b087 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dbab19e2797462ba71dcb40202eac97', 'request_id': 'req-cdc92a21-e52f-4d7e-a079-95ee5a53b087'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:47.726732 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-52981288-ac3e-44dc-aaab-655b61c80bea None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:47.727045 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-cdc92a21-e52f-4d7e-a079-95ee5a53b087 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dbab19e2797462ba71dcb40202eac97', 'request_id': 'req-cdc92a21-e52f-4d7e-a079-95ee5a53b087'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:47.727145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-52981288-ac3e-44dc-aaab-655b61c80bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:47.822042 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-cdc92a21-e52f-4d7e-a079-95ee5a53b087 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dbab19e2797462ba71dcb40202eac97', 'request_id': 'req-cdc92a21-e52f-4d7e-a079-95ee5a53b087'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:47.840128 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-cdc92a21-e52f-4d7e-a079-95ee5a53b087 admin admin] The token cache is being invalidate because project 1dbab19e2797462ba71dcb40202eac97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:34:47.842313 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-cdc92a21-e52f-4d7e-a079-95ee5a53b087 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:47.843979 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-cdc92a21-e52f-4d7e-a079-95ee5a53b087 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:47.849139 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 896/1774] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:34:47 2022] DELETE /identity/v3/projects/1dbab19e2797462ba71dcb40202eac97 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:47.855734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfcd1b-ed12-4183-9b1b-e508dfc8ad15 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:47.856289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfcd1b-ed12-4183-9b1b-e508dfc8ad15 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:47.859473 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfcd1b-ed12-4183-9b1b-e508dfc8ad15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:47.878219 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 879/1775] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:34:47.886300 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf827a-499a-42b1-8203-12d7f6176783 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:47.888665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-e9cfcd1b-ed12-4183-9b1b-e508dfc8ad15 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:47.899588 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf827a-499a-42b1-8203-12d7f6176783 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:47.899588 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf827a-499a-42b1-8203-12d7f6176783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:47.899876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e9cfcd1b-ed12-4183-9b1b-e508dfc8ad15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:47.910660 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 897/1776] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:47.920099 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c58802-f2a9-4e94-874f-11a5f134ae76 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:47.920653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c58802-f2a9-4e94-874f-11a5f134ae76 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:47.921150 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c58802-f2a9-4e94-874f-11a5f134ae76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:47.930851 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-bebf827a-499a-42b1-8203-12d7f6176783 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:47.935170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-bebf827a-499a-42b1-8203-12d7f6176783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:47.947925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-09c58802-f2a9-4e94-874f-11a5f134ae76 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:47.950164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-09c58802-f2a9-4e94-874f-11a5f134ae76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:47.954023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 880/1777] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:47.958571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf827a-499a-42b1-8203-12d7f6176783 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:47.962708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 898/1778] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:47.968788 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c58802-f2a9-4e94-874f-11a5f134ae76 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:47.978437 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf827a-499a-42b1-8203-12d7f6176783 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:47.985881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c58802-f2a9-4e94-874f-11a5f134ae76 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:47.993195 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6bb377-0453-4c9a-8e35-8e51c6292e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:47.993673 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0e5e7-d77c-4f21-9e1b-bc7c1c1e8415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:47.994233 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bb377-0453-4c9a-8e35-8e51c6292e8a admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:47.994444 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bb377-0453-4c9a-8e35-8e51c6292e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:47.994745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bb377-0453-4c9a-8e35-8e51c6292e8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:47.995631 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6bb377-0453-4c9a-8e35-8e51c6292e8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:47.995905 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6bb377-0453-4c9a-8e35-8e51c6292e8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:47.996526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0e5e7-d77c-4f21-9e1b-bc7c1c1e8415 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:47.997774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6bb377-0453-4c9a-8e35-8e51c6292e8a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:47.998316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0e5e7-d77c-4f21-9e1b-bc7c1c1e8415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:47.998928 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0e5e7-d77c-4f21-9e1b-bc7c1c1e8415 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:48.000622 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0e5e7-d77c-4f21-9e1b-bc7c1c1e8415 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:48.001088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0e5e7-d77c-4f21-9e1b-bc7c1c1e8415 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:48.004303 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0e5e7-d77c-4f21-9e1b-bc7c1c1e8415 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:48.015775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 881/1779] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:34:48.022548 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6bb377-0453-4c9a-8e35-8e51c6292e8a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:48.028940 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 899/1780] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:34:48.036472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0e5e7-d77c-4f21-9e1b-bc7c1c1e8415 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:48.051411 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6bb377-0453-4c9a-8e35-8e51c6292e8a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:48.053207 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0e5e7-d77c-4f21-9e1b-bc7c1c1e8415 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:48.064822 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21eb1e-1609-4d97-9569-717dcf51caf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:48.064822 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21eb1e-1609-4d97-9569-717dcf51caf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:48.065152 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867606a1-3066-403c-b2f5-73cd7522a979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:48.065254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21eb1e-1609-4d97-9569-717dcf51caf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:48.065254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21eb1e-1609-4d97-9569-717dcf51caf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:48.065406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867606a1-3066-403c-b2f5-73cd7522a979 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:48.065406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867606a1-3066-403c-b2f5-73cd7522a979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:48.065570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21eb1e-1609-4d97-9569-717dcf51caf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:48.065652 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867606a1-3066-403c-b2f5-73cd7522a979 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:48.065652 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867606a1-3066-403c-b2f5-73cd7522a979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:48.065792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21eb1e-1609-4d97-9569-717dcf51caf5 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:48.065883 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867606a1-3066-403c-b2f5-73cd7522a979 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:48.235422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 900/1781] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:48 2022] POST /identity/v3/projects => generated 379 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:48.241717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21eb1e-1609-4d97-9569-717dcf51caf5 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:48.242148 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 882/1782] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:48 2022] POST /identity/v3/projects => generated 363 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:48.246236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867606a1-3066-403c-b2f5-73cd7522a979 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:48.412794 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21eb1e-1609-4d97-9569-717dcf51caf5 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:48.516934 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867606a1-3066-403c-b2f5-73cd7522a979 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:48.522730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46666555-f0ed-4137-bfd7-a328e21ea2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:48.523622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46666555-f0ed-4137-bfd7-a328e21ea2af admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:48.524124 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46666555-f0ed-4137-bfd7-a328e21ea2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:48.524547 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46666555-f0ed-4137-bfd7-a328e21ea2af admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:48.525406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46666555-f0ed-4137-bfd7-a328e21ea2af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:48.527630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46666555-f0ed-4137-bfd7-a328e21ea2af admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:48.547028 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897d201-5317-4c42-930c-91eb3f6e098c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:48.549307 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897d201-5317-4c42-930c-91eb3f6e098c admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:48.549599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897d201-5317-4c42-930c-91eb3f6e098c admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:48.550894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897d201-5317-4c42-930c-91eb3f6e098c admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:48.552674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897d201-5317-4c42-930c-91eb3f6e098c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:48.557136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897d201-5317-4c42-930c-91eb3f6e098c admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:48.681298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 883/1783] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:48 2022] POST /identity/v3/users => generated 282 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:48.681298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46666555-f0ed-4137-bfd7-a328e21ea2af admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:48.694755 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46666555-f0ed-4137-bfd7-a328e21ea2af admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:48.701836 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2cc84-1c76-4ad1-91ed-7c52adb538b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:48.702648 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2cc84-1c76-4ad1-91ed-7c52adb538b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:48.703025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2cc84-1c76-4ad1-91ed-7c52adb538b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:48.703373 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2cc84-1c76-4ad1-91ed-7c52adb538b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:48.704040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2cc84-1c76-4ad1-91ed-7c52adb538b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:48.706181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2cc84-1c76-4ad1-91ed-7c52adb538b2 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:48.726841 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 884/1784] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:34:48.730850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 901/1785] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:48 2022] POST /identity/v3/users => generated 290 bytes in 489 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:48.743354 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2cc84-1c76-4ad1-91ed-7c52adb538b2 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:48.749794 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897d201-5317-4c42-930c-91eb3f6e098c admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:48.758061 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2cc84-1c76-4ad1-91ed-7c52adb538b2 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:48.777524 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9713ae8-f0e2-4135-b443-9bfc9fb23fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:48.778047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897d201-5317-4c42-930c-91eb3f6e098c admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:48.778158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9713ae8-f0e2-4135-b443-9bfc9fb23fdd admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:48.778449 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9713ae8-f0e2-4135-b443-9bfc9fb23fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:48.778725 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9713ae8-f0e2-4135-b443-9bfc9fb23fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:48.779484 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9713ae8-f0e2-4135-b443-9bfc9fb23fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:48.781622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9713ae8-f0e2-4135-b443-9bfc9fb23fdd admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:48.790631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01665b-766d-4161-af25-b60273666a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:48.791832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01665b-766d-4161-af25-b60273666a0b admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:48.795989 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01665b-766d-4161-af25-b60273666a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:48.795989 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01665b-766d-4161-af25-b60273666a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:48.796770 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01665b-766d-4161-af25-b60273666a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:48.803803 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01665b-766d-4161-af25-b60273666a0b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:48.812603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 885/1786] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:48.822281 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9713ae8-f0e2-4135-b443-9bfc9fb23fdd admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:48.831268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 902/1787] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:48.841539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9713ae8-f0e2-4135-b443-9bfc9fb23fdd admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:48.846225 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01665b-766d-4161-af25-b60273666a0b admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:48.856818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53fddd-0a25-445c-89c2-ffae765e34dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:48.857236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53fddd-0a25-445c-89c2-ffae765e34dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:48.857522 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53fddd-0a25-445c-89c2-ffae765e34dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:48.868259 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53fddd-0a25-445c-89c2-ffae765e34dd admin admin] PATH_INFO: `/v3/projects/b6f80a75c9c54a39a34f66535a537e15/users/274a4ef2f20c495a82093bca5719aec9/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:48.868391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01665b-766d-4161-af25-b60273666a0b admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:48.868391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f2530-5311-4008-92d1-0b78d766be1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:48.880426 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c53fddd-0a25-445c-89c2-ffae765e34dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f80a75c9c54a39a34f66535a537e15, user_id=274a4ef2f20c495a82093bca5719aec9, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:48.883385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c53fddd-0a25-445c-89c2-ffae765e34dd admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:48.883557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f2530-5311-4008-92d1-0b78d766be1c admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:48.884797 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f2530-5311-4008-92d1-0b78d766be1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:48.885093 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f2530-5311-4008-92d1-0b78d766be1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:48.887462 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f2530-5311-4008-92d1-0b78d766be1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:48.889102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f2530-5311-4008-92d1-0b78d766be1c admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:48.903686 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 903/1788] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:34:48.919106 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f2530-5311-4008-92d1-0b78d766be1c admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:48.925580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f2530-5311-4008-92d1-0b78d766be1c admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:48.932550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63044a4b-81f7-481f-98cb-d70a58755f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:48.933223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63044a4b-81f7-481f-98cb-d70a58755f8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:48.933625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63044a4b-81f7-481f-98cb-d70a58755f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:48.933942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63044a4b-81f7-481f-98cb-d70a58755f8e admin admin] PATH_INFO: `/v3/projects/f7ae2eb1a4cb4667b15359aabbb255e5/users/8abe5dcde3cf4471b180b0199874bc06/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:48.959307 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63044a4b-81f7-481f-98cb-d70a58755f8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7ae2eb1a4cb4667b15359aabbb255e5, user_id=8abe5dcde3cf4471b180b0199874bc06, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:48.963639 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63044a4b-81f7-481f-98cb-d70a58755f8e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:49.015721 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 886/1789] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:48 2022] PUT /identity/v3/projects/b6f80a75c9c54a39a34f66535a537e15/users/274a4ef2f20c495a82093bca5719aec9/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:49.028463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402219f-f85b-401a-a99a-78c64a1efaa0 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:49.029072 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402219f-f85b-401a-a99a-78c64a1efaa0 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:49.029635 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402219f-f85b-401a-a99a-78c64a1efaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:49.082949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-7402219f-f85b-401a-a99a-78c64a1efaa0 None None] MFA Rules not processed for user `274a4ef2f20c495a82093bca5719aec9`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:49.085337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-7402219f-f85b-401a-a99a-78c64a1efaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:49.104038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 904/1790] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:48 2022] PUT /identity/v3/projects/f7ae2eb1a4cb4667b15359aabbb255e5/users/8abe5dcde3cf4471b180b0199874bc06/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:49.116933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfec503-6a53-4913-94d7-56170a44dcfa None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:49.117656 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfec503-6a53-4913-94d7-56170a44dcfa None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:49.117656 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfec503-6a53-4913-94d7-56170a44dcfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:49.193572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-dcfec503-6a53-4913-94d7-56170a44dcfa None None] MFA Rules not processed for user `8abe5dcde3cf4471b180b0199874bc06`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:49.194748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-dcfec503-6a53-4913-94d7-56170a44dcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:49.207525 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 887/1791] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:34:49.211351 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7402219f-f85b-401a-a99a-78c64a1efaa0 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:49.300928 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 905/1792] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:49.305282 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfec503-6a53-4913-94d7-56170a44dcfa None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:49.348211 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfec503-6a53-4913-94d7-56170a44dcfa None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:49.348532 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7402219f-f85b-401a-a99a-78c64a1efaa0 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:49.364492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4a3f8-814f-4422-8489-386015d8fa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:49.365412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4a3f8-814f-4422-8489-386015d8fa60 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:49.365843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4a3f8-814f-4422-8489-386015d8fa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:49.366258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4a3f8-814f-4422-8489-386015d8fa60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:49.367230 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4a3f8-814f-4422-8489-386015d8fa60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:49.373861 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4a3f8-814f-4422-8489-386015d8fa60 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:49.386097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d318df-2d2e-44fa-adc1-4036b3dec92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:49.386737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d318df-2d2e-44fa-adc1-4036b3dec92b admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:49.387200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d318df-2d2e-44fa-adc1-4036b3dec92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:49.387574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d318df-2d2e-44fa-adc1-4036b3dec92b admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:49.392367 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d318df-2d2e-44fa-adc1-4036b3dec92b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:49.413461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d318df-2d2e-44fa-adc1-4036b3dec92b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:49.473477 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:49.537930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 888/1793] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:49 2022] POST /identity/v3/projects => generated 365 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:49.543037 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4a3f8-814f-4422-8489-386015d8fa60 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:49.578299 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 906/1794] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:49 2022] POST /identity/v3/projects => generated 379 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:49.583089 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d318df-2d2e-44fa-adc1-4036b3dec92b admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:49.638687 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:49.737717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4a3f8-814f-4422-8489-386015d8fa60 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:49.779560 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d318df-2d2e-44fa-adc1-4036b3dec92b admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:49.790541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc8d40-c4aa-47ca-b541-526bc9f198ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:49.793602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc8d40-c4aa-47ca-b541-526bc9f198ae admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:49.794245 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc8d40-c4aa-47ca-b541-526bc9f198ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:49.795811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc8d40-c4aa-47ca-b541-526bc9f198ae admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:49.797402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc8d40-c4aa-47ca-b541-526bc9f198ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:49.805048 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc8d40-c4aa-47ca-b541-526bc9f198ae admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:49.830278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd70e42-80a0-4835-bb44-88a6bdacd4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:49.830278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd70e42-80a0-4835-bb44-88a6bdacd4b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:49.830278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd70e42-80a0-4835-bb44-88a6bdacd4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:49.830278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd70e42-80a0-4835-bb44-88a6bdacd4b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:49.836147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd70e42-80a0-4835-bb44-88a6bdacd4b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:49.836147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd70e42-80a0-4835-bb44-88a6bdacd4b6 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:50.012159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 907/1795] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:49 2022] POST /identity/v3/users => generated 296 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:34:50.019978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 889/1796] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:49 2022] POST /identity/v3/users => generated 289 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:50.021751 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc8d40-c4aa-47ca-b541-526bc9f198ae admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:50.022491 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd70e42-80a0-4835-bb44-88a6bdacd4b6 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:50.057724 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd70e42-80a0-4835-bb44-88a6bdacd4b6 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:50.068599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc8d40-c4aa-47ca-b541-526bc9f198ae admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:50.084610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0140506-fb45-4cf7-bdae-9d56aff033c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:50.086148 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0140506-fb45-4cf7-bdae-9d56aff033c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:50.087255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0140506-fb45-4cf7-bdae-9d56aff033c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:50.087689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0140506-fb45-4cf7-bdae-9d56aff033c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:50.088860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0a9d9-6cb7-4624-8ff4-01c61c97a0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:50.089600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0140506-fb45-4cf7-bdae-9d56aff033c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:50.091546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0a9d9-6cb7-4624-8ff4-01c61c97a0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:50.093292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0a9d9-6cb7-4624-8ff4-01c61c97a0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:50.093724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0a9d9-6cb7-4624-8ff4-01c61c97a0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:50.094552 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0a9d9-6cb7-4624-8ff4-01c61c97a0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:50.102590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0140506-fb45-4cf7-bdae-9d56aff033c8 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:50.107483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0a9d9-6cb7-4624-8ff4-01c61c97a0b1 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:50.117191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 890/1797] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:50.118810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0140506-fb45-4cf7-bdae-9d56aff033c8 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:50.131610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0140506-fb45-4cf7-bdae-9d56aff033c8 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:50.138299 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ee28f-9a70-48d5-87ce-6283418e7ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:50.139855 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ee28f-9a70-48d5-87ce-6283418e7ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:50.139855 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ee28f-9a70-48d5-87ce-6283418e7ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:50.141457 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ee28f-9a70-48d5-87ce-6283418e7ec4 admin admin] PATH_INFO: `/v3/projects/c83f4661b4364faab2e3b82ab3f0d7d3/users/b2a6563241124029aa7440f3369b0383/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:50.154221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 908/1798] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:50.155328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0a9d9-6cb7-4624-8ff4-01c61c97a0b1 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:50.179448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0a9d9-6cb7-4624-8ff4-01c61c97a0b1 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:50.186617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27caeb0-ff96-48e2-aa98-3f5dac87dc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:50.187515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27caeb0-ff96-48e2-aa98-3f5dac87dc28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:50.187741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27caeb0-ff96-48e2-aa98-3f5dac87dc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:50.188095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ee28f-9a70-48d5-87ce-6283418e7ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83f4661b4364faab2e3b82ab3f0d7d3, user_id=b2a6563241124029aa7440f3369b0383, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:50.189942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27caeb0-ff96-48e2-aa98-3f5dac87dc28 admin admin] PATH_INFO: `/v3/projects/af5cec233bd443cd9846dbe33e4d76b8/users/088863963d8a42ba894e1d05329630b9/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:50.192840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ee28f-9a70-48d5-87ce-6283418e7ec4 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:50.238657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27caeb0-ff96-48e2-aa98-3f5dac87dc28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af5cec233bd443cd9846dbe33e4d76b8, user_id=088863963d8a42ba894e1d05329630b9, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:50.241397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27caeb0-ff96-48e2-aa98-3f5dac87dc28 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:50.307645 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 891/1799] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:50 2022] PUT /identity/v3/projects/c83f4661b4364faab2e3b82ab3f0d7d3/users/b2a6563241124029aa7440f3369b0383/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:34:50.308797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ee28f-9a70-48d5-87ce-6283418e7ec4 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:50.395660 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 909/1800] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:50 2022] PUT /identity/v3/projects/af5cec233bd443cd9846dbe33e4d76b8/users/088863963d8a42ba894e1d05329630b9/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:50.396614 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27caeb0-ff96-48e2-aa98-3f5dac87dc28 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:50.498998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ee28f-9a70-48d5-87ce-6283418e7ec4 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:50.577954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27caeb0-ff96-48e2-aa98-3f5dac87dc28 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:50.589867 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d14de9-51c0-457d-8746-2bb5d1f95879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:50.590581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d14de9-51c0-457d-8746-2bb5d1f95879 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:50.590896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d14de9-51c0-457d-8746-2bb5d1f95879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:50.591310 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d14de9-51c0-457d-8746-2bb5d1f95879 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:50.598773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d14de9-51c0-457d-8746-2bb5d1f95879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:50.599531 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d14de9-51c0-457d-8746-2bb5d1f95879 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:50.626190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 910/1801] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:50.634902 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d14de9-51c0-457d-8746-2bb5d1f95879 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:50.646177 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d14de9-51c0-457d-8746-2bb5d1f95879 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:50.665331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c42460-201f-42f4-8737-79f2c5f6b8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:50.666308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c42460-201f-42f4-8737-79f2c5f6b8b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:50.666768 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c42460-201f-42f4-8737-79f2c5f6b8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:50.667286 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c42460-201f-42f4-8737-79f2c5f6b8b6 admin admin] PATH_INFO: `/v3/projects/af5cec233bd443cd9846dbe33e4d76b8/users/088863963d8a42ba894e1d05329630b9/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:50.670501 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c42460-201f-42f4-8737-79f2c5f6b8b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af5cec233bd443cd9846dbe33e4d76b8, user_id=088863963d8a42ba894e1d05329630b9, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:50.673834 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c42460-201f-42f4-8737-79f2c5f6b8b6 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:50.698156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6257c92-224f-49e1-8d50-39fbffe9a866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:50.698156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6257c92-224f-49e1-8d50-39fbffe9a866 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:50.698156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6257c92-224f-49e1-8d50-39fbffe9a866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:50.698156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6257c92-224f-49e1-8d50-39fbffe9a866 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:50.705996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6257c92-224f-49e1-8d50-39fbffe9a866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:50.705996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6257c92-224f-49e1-8d50-39fbffe9a866 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:50.706160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 911/1802] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:50 2022] PUT /identity/v3/projects/af5cec233bd443cd9846dbe33e4d76b8/users/088863963d8a42ba894e1d05329630b9/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:34:50.706160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6c778-e400-4261-b7a2-19d476153760 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:50.706160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6c778-e400-4261-b7a2-19d476153760 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:50.706160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6c778-e400-4261-b7a2-19d476153760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:50.724824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 892/1803] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:50.727806 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6257c92-224f-49e1-8d50-39fbffe9a866 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:50.743470 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-e9a6c778-e400-4261-b7a2-19d476153760 None None] MFA Rules not processed for user `088863963d8a42ba894e1d05329630b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:50.745066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e9a6c778-e400-4261-b7a2-19d476153760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:50.754314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6257c92-224f-49e1-8d50-39fbffe9a866 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:50.879711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 912/1804] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:50 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:34:50.893923 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6c778-e400-4261-b7a2-19d476153760 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:50.895816 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e9a6c778-e400-4261-b7a2-19d476153760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:50.898350 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a613b4a-5c8f-4e3e-8e2c-0f077443e5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:50.899332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a613b4a-5c8f-4e3e-8e2c-0f077443e5a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:50.899778 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a613b4a-5c8f-4e3e-8e2c-0f077443e5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:50.900077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a613b4a-5c8f-4e3e-8e2c-0f077443e5a8 admin admin] PATH_INFO: `/v3/projects/c83f4661b4364faab2e3b82ab3f0d7d3/users/b2a6563241124029aa7440f3369b0383/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:50.905318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a613b4a-5c8f-4e3e-8e2c-0f077443e5a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83f4661b4364faab2e3b82ab3f0d7d3, user_id=b2a6563241124029aa7440f3369b0383, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:50.908850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a613b4a-5c8f-4e3e-8e2c-0f077443e5a8 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:50.937099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 893/1805] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:50 2022] PUT /identity/v3/projects/c83f4661b4364faab2e3b82ab3f0d7d3/users/b2a6563241124029aa7440f3369b0383/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:50.938829 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795c139-f963-4778-9bd8-35e136d6d53f None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:50.939138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795c139-f963-4778-9bd8-35e136d6d53f None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:50.940019 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795c139-f963-4778-9bd8-35e136d6d53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:50.982603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-7795c139-f963-4778-9bd8-35e136d6d53f None None] MFA Rules not processed for user `b2a6563241124029aa7440f3369b0383`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:50.984860 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-7795c139-f963-4778-9bd8-35e136d6d53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:50.990358 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6c778-e400-4261-b7a2-19d476153760 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:51.059297 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e03623-0e70-466e-94c3-6c3e3619ae59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:51.059972 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e03623-0e70-466e-94c3-6c3e3619ae59 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:51.060379 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e03623-0e70-466e-94c3-6c3e3619ae59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:51.060737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e03623-0e70-466e-94c3-6c3e3619ae59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:51.082703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 894/1806] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:34:51.126549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795c139-f963-4778-9bd8-35e136d6d53f None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:51.128613 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-7795c139-f963-4778-9bd8-35e136d6d53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:51.141004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795c139-f963-4778-9bd8-35e136d6d53f None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:51.145888 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac67be4-4cfb-42b9-b70c-afbe665e2337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:51.146512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac67be4-4cfb-42b9-b70c-afbe665e2337 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:51.147005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac67be4-4cfb-42b9-b70c-afbe665e2337 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:51.147129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac67be4-4cfb-42b9-b70c-afbe665e2337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:51.148350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e03623-0e70-466e-94c3-6c3e3619ae59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:51.150273 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e03623-0e70-466e-94c3-6c3e3619ae59 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:51.161386 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac67be4-4cfb-42b9-b70c-afbe665e2337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:51.162366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 913/1807] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:50 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:34:51.162866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac67be4-4cfb-42b9-b70c-afbe665e2337 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:51.174527 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 895/1808] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:34:51.880381 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e03623-0e70-466e-94c3-6c3e3619ae59 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:51.948571 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e03623-0e70-466e-94c3-6c3e3619ae59 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:51.953672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff85bd-6e13-41b2-be16-5565d627388f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:51.958617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff85bd-6e13-41b2-be16-5565d627388f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:51.958617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff85bd-6e13-41b2-be16-5565d627388f admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:51.958617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff85bd-6e13-41b2-be16-5565d627388f admin admin] PATH_INFO: `/v3/users/8abe5dcde3cf4471b180b0199874bc06` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:51.958617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff85bd-6e13-41b2-be16-5565d627388f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8abe5dcde3cf4471b180b0199874bc06)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:51.959078 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff85bd-6e13-41b2-be16-5565d627388f admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:52.251467 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b8ff85bd-6e13-41b2-be16-5565d627388f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8abe5dcde3cf4471b180b0199874bc06'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:52.339987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b8ff85bd-6e13-41b2-be16-5565d627388f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8abe5dcde3cf4471b180b0199874bc06'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:52.346365 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac67be4-4cfb-42b9-b70c-afbe665e2337 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:52.356740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac67be4-4cfb-42b9-b70c-afbe665e2337 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:52.360620 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b8ff85bd-6e13-41b2-be16-5565d627388f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8abe5dcde3cf4471b180b0199874bc06'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:52.361900 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b8ff85bd-6e13-41b2-be16-5565d627388f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8abe5dcde3cf4471b180b0199874bc06'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:52.366923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ff466-7c8a-4e69-b53e-91305d087a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:52.366923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ff466-7c8a-4e69-b53e-91305d087a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:52.366923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ff466-7c8a-4e69-b53e-91305d087a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:52.366923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ff466-7c8a-4e69-b53e-91305d087a03 admin admin] PATH_INFO: `/v3/users/274a4ef2f20c495a82093bca5719aec9` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:52.367281 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ff466-7c8a-4e69-b53e-91305d087a03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=274a4ef2f20c495a82093bca5719aec9)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:52.367281 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ff466-7c8a-4e69-b53e-91305d087a03 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:52.369081 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b8ff85bd-6e13-41b2-be16-5565d627388f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8abe5dcde3cf4471b180b0199874bc06'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:52.375146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 914/1809] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:34:51 2022] DELETE /identity/v3/users/8abe5dcde3cf4471b180b0199874bc06 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:52.379459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cde883-ea74-4e6d-b758-ecb86a9645fb None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:52.379754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cde883-ea74-4e6d-b758-ecb86a9645fb None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:52.380052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cde883-ea74-4e6d-b758-ecb86a9645fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:52.400388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-34cde883-ea74-4e6d-b758-ecb86a9645fb None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:52.401693 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-34cde883-ea74-4e6d-b758-ecb86a9645fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:52.468259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 915/1810] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:52.478987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cde883-ea74-4e6d-b758-ecb86a9645fb None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:52.483218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-34cde883-ea74-4e6d-b758-ecb86a9645fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:52.554974 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cde883-ea74-4e6d-b758-ecb86a9645fb None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:52.558901 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b74bec-721e-4b1a-bb8d-9335e6747cc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:52.559730 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b74bec-721e-4b1a-bb8d-9335e6747cc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:52.560010 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b74bec-721e-4b1a-bb8d-9335e6747cc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:52.560279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b74bec-721e-4b1a-bb8d-9335e6747cc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:52.570321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b74bec-721e-4b1a-bb8d-9335e6747cc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:52.572486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b74bec-721e-4b1a-bb8d-9335e6747cc5 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:52.585796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 916/1811] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:52.601556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-968ff466-7c8a-4e69-b53e-91305d087a03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '274a4ef2f20c495a82093bca5719aec9'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:52.726284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:52.727020 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-968ff466-7c8a-4e69-b53e-91305d087a03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '274a4ef2f20c495a82093bca5719aec9'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:52.727782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-968ff466-7c8a-4e69-b53e-91305d087a03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '274a4ef2f20c495a82093bca5719aec9'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:52.740224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-968ff466-7c8a-4e69-b53e-91305d087a03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '274a4ef2f20c495a82093bca5719aec9'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:52.747239 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-968ff466-7c8a-4e69-b53e-91305d087a03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '274a4ef2f20c495a82093bca5719aec9'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:52.753786 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 896/1812] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:34:52 2022] DELETE /identity/v3/users/274a4ef2f20c495a82093bca5719aec9 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:52.758185 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe82cd8-3df8-4a73-a276-c94318881297 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:52.758472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe82cd8-3df8-4a73-a276-c94318881297 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:52.758748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe82cd8-3df8-4a73-a276-c94318881297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:52.779360 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-3fe82cd8-3df8-4a73-a276-c94318881297 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:52.780599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-3fe82cd8-3df8-4a73-a276-c94318881297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:52.843036 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 917/1813] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:52.853397 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ff466-7c8a-4e69-b53e-91305d087a03 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:52.854767 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-968ff466-7c8a-4e69-b53e-91305d087a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:52.913581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:52.918719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ff466-7c8a-4e69-b53e-91305d087a03 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:52.922481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a12970-c0c2-4a04-921e-b7db845cbbad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:52.923597 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a12970-c0c2-4a04-921e-b7db845cbbad service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:52.923597 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a12970-c0c2-4a04-921e-b7db845cbbad service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:52.924450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a12970-c0c2-4a04-921e-b7db845cbbad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:52.937840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a12970-c0c2-4a04-921e-b7db845cbbad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:52.939347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a12970-c0c2-4a04-921e-b7db845cbbad service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:52.949612 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 897/1814] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:53.036304 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe82cd8-3df8-4a73-a276-c94318881297 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:53.037955 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-3fe82cd8-3df8-4a73-a276-c94318881297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:53.051141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe82cd8-3df8-4a73-a276-c94318881297 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:53.056052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819def82-5f8c-4b50-b805-f55bee98f2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:53.058547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819def82-5f8c-4b50-b805-f55bee98f2f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:53.058547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819def82-5f8c-4b50-b805-f55bee98f2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:53.058547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819def82-5f8c-4b50-b805-f55bee98f2f2 admin admin] PATH_INFO: `/v3/projects/f7ae2eb1a4cb4667b15359aabbb255e5` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:53.060135 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819def82-5f8c-4b50-b805-f55bee98f2f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7ae2eb1a4cb4667b15359aabbb255e5)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:53.062184 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819def82-5f8c-4b50-b805-f55bee98f2f2 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:53.203438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-819def82-5f8c-4b50-b805-f55bee98f2f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7ae2eb1a4cb4667b15359aabbb255e5', 'request_id': 'req-819def82-5f8c-4b50-b805-f55bee98f2f2'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:53.210677 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-819def82-5f8c-4b50-b805-f55bee98f2f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7ae2eb1a4cb4667b15359aabbb255e5', 'request_id': 'req-819def82-5f8c-4b50-b805-f55bee98f2f2'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:53.333832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-819def82-5f8c-4b50-b805-f55bee98f2f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7ae2eb1a4cb4667b15359aabbb255e5', 'request_id': 'req-819def82-5f8c-4b50-b805-f55bee98f2f2'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:53.345182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-819def82-5f8c-4b50-b805-f55bee98f2f2 admin admin] The token cache is being invalidate because project f7ae2eb1a4cb4667b15359aabbb255e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:34:53.345548 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-819def82-5f8c-4b50-b805-f55bee98f2f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:53.346273 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-819def82-5f8c-4b50-b805-f55bee98f2f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:53.348038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 918/1815] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:34:53 2022] DELETE /identity/v3/projects/f7ae2eb1a4cb4667b15359aabbb255e5 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:53.353072 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a12970-c0c2-4a04-921e-b7db845cbbad service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:53.363506 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a12970-c0c2-4a04-921e-b7db845cbbad service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:53.418169 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d536c24-90ae-4b12-a3b1-5c0586fc21a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:53.418809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d536c24-90ae-4b12-a3b1-5c0586fc21a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:53.419110 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d536c24-90ae-4b12-a3b1-5c0586fc21a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:53.419383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d536c24-90ae-4b12-a3b1-5c0586fc21a3 admin admin] PATH_INFO: `/v3/users/b2a6563241124029aa7440f3369b0383` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:53.420511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d536c24-90ae-4b12-a3b1-5c0586fc21a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2a6563241124029aa7440f3369b0383)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:53.422437 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d536c24-90ae-4b12-a3b1-5c0586fc21a3 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:53.512480 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819def82-5f8c-4b50-b805-f55bee98f2f2 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:53.513969 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-819def82-5f8c-4b50-b805-f55bee98f2f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:53.529682 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819def82-5f8c-4b50-b805-f55bee98f2f2 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:53.535911 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3898bab-ae42-4c61-86e3-50b5e94ee26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:53.536883 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3898bab-ae42-4c61-86e3-50b5e94ee26b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:53.537605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3898bab-ae42-4c61-86e3-50b5e94ee26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:53.538044 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3898bab-ae42-4c61-86e3-50b5e94ee26b admin admin] PATH_INFO: `/v3/projects/b6f80a75c9c54a39a34f66535a537e15` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:53.539530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3898bab-ae42-4c61-86e3-50b5e94ee26b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6f80a75c9c54a39a34f66535a537e15)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:53.542942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3898bab-ae42-4c61-86e3-50b5e94ee26b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:53.686954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3d536c24-90ae-4b12-a3b1-5c0586fc21a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2a6563241124029aa7440f3369b0383'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:53.701746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e3898bab-ae42-4c61-86e3-50b5e94ee26b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6f80a75c9c54a39a34f66535a537e15', 'request_id': 'req-e3898bab-ae42-4c61-86e3-50b5e94ee26b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:53.724557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e3898bab-ae42-4c61-86e3-50b5e94ee26b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6f80a75c9c54a39a34f66535a537e15', 'request_id': 'req-e3898bab-ae42-4c61-86e3-50b5e94ee26b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:53.754016 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3d536c24-90ae-4b12-a3b1-5c0586fc21a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2a6563241124029aa7440f3369b0383'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:53.763395 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3d536c24-90ae-4b12-a3b1-5c0586fc21a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2a6563241124029aa7440f3369b0383'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:53.768392 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3d536c24-90ae-4b12-a3b1-5c0586fc21a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2a6563241124029aa7440f3369b0383'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:53.775435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3d536c24-90ae-4b12-a3b1-5c0586fc21a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2a6563241124029aa7440f3369b0383'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:53.781480 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 898/1816] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:34:53 2022] DELETE /identity/v3/users/b2a6563241124029aa7440f3369b0383 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:34:53.842758 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e3898bab-ae42-4c61-86e3-50b5e94ee26b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6f80a75c9c54a39a34f66535a537e15', 'request_id': 'req-e3898bab-ae42-4c61-86e3-50b5e94ee26b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:53.855613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e3898bab-ae42-4c61-86e3-50b5e94ee26b admin admin] The token cache is being invalidate because project b6f80a75c9c54a39a34f66535a537e15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:34:53.855917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e3898bab-ae42-4c61-86e3-50b5e94ee26b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:53.856844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e3898bab-ae42-4c61-86e3-50b5e94ee26b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:53.858369 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 919/1817] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:34:53 2022] DELETE /identity/v3/projects/b6f80a75c9c54a39a34f66535a537e15 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:34:53.868364 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d536c24-90ae-4b12-a3b1-5c0586fc21a3 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:53.870001 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-3d536c24-90ae-4b12-a3b1-5c0586fc21a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:53.943694 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d536c24-90ae-4b12-a3b1-5c0586fc21a3 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:53.948232 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae18876-b6dc-4470-b0d9-012bd3fc4a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:53.948862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae18876-b6dc-4470-b0d9-012bd3fc4a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:53.949141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae18876-b6dc-4470-b0d9-012bd3fc4a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:53.949413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae18876-b6dc-4470-b0d9-012bd3fc4a1d admin admin] PATH_INFO: `/v3/users/088863963d8a42ba894e1d05329630b9` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:53.950519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae18876-b6dc-4470-b0d9-012bd3fc4a1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=088863963d8a42ba894e1d05329630b9)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:53.952455 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae18876-b6dc-4470-b0d9-012bd3fc4a1d admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:54.272486 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-7ae18876-b6dc-4470-b0d9-012bd3fc4a1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '088863963d8a42ba894e1d05329630b9'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:54.320738 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3898bab-ae42-4c61-86e3-50b5e94ee26b admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:54.323981 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e3898bab-ae42-4c61-86e3-50b5e94ee26b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:54.431170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-7ae18876-b6dc-4470-b0d9-012bd3fc4a1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '088863963d8a42ba894e1d05329630b9'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:54.432437 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-7ae18876-b6dc-4470-b0d9-012bd3fc4a1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '088863963d8a42ba894e1d05329630b9'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:54.448093 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3898bab-ae42-4c61-86e3-50b5e94ee26b admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:54.451292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-7ae18876-b6dc-4470-b0d9-012bd3fc4a1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '088863963d8a42ba894e1d05329630b9'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:54.460414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92744e8d-7138-448c-9062-28a6a327bb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:54.461250 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92744e8d-7138-448c-9062-28a6a327bb97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:54.461728 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92744e8d-7138-448c-9062-28a6a327bb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:54.462696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92744e8d-7138-448c-9062-28a6a327bb97 admin admin] PATH_INFO: `/v3/projects/c83f4661b4364faab2e3b82ab3f0d7d3` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:54.463131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-7ae18876-b6dc-4470-b0d9-012bd3fc4a1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '088863963d8a42ba894e1d05329630b9'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:54.466154 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92744e8d-7138-448c-9062-28a6a327bb97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c83f4661b4364faab2e3b82ab3f0d7d3)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:54.469827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92744e8d-7138-448c-9062-28a6a327bb97 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:54.474580 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 899/1818] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:34:53 2022] DELETE /identity/v3/users/088863963d8a42ba894e1d05329630b9 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:54.608969 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-92744e8d-7138-448c-9062-28a6a327bb97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c83f4661b4364faab2e3b82ab3f0d7d3', 'request_id': 'req-92744e8d-7138-448c-9062-28a6a327bb97'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:54.616503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-92744e8d-7138-448c-9062-28a6a327bb97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c83f4661b4364faab2e3b82ab3f0d7d3', 'request_id': 'req-92744e8d-7138-448c-9062-28a6a327bb97'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:54.718542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-92744e8d-7138-448c-9062-28a6a327bb97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c83f4661b4364faab2e3b82ab3f0d7d3', 'request_id': 'req-92744e8d-7138-448c-9062-28a6a327bb97'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:54.735590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-92744e8d-7138-448c-9062-28a6a327bb97 admin admin] The token cache is being invalidate because project c83f4661b4364faab2e3b82ab3f0d7d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:34:54.736826 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-92744e8d-7138-448c-9062-28a6a327bb97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:54.737621 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-92744e8d-7138-448c-9062-28a6a327bb97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:54.739919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 920/1819] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:34:54 2022] DELETE /identity/v3/projects/c83f4661b4364faab2e3b82ab3f0d7d3 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:54.744886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e19e7-795b-4351-99f4-9bdd40be4a9b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:54.745181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e19e7-795b-4351-99f4-9bdd40be4a9b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:54.745464 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e19e7-795b-4351-99f4-9bdd40be4a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:54.766254 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-8f2e19e7-795b-4351-99f4-9bdd40be4a9b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:54.767600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8f2e19e7-795b-4351-99f4-9bdd40be4a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:54.827209 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 900/1820] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:54.837460 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b590b-801c-4266-b483-035c39c58aa8 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:54.838159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b590b-801c-4266-b483-035c39c58aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:54.838814 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b590b-801c-4266-b483-035c39c58aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:54.859257 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-105b590b-801c-4266-b483-035c39c58aa8 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:54.860478 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-105b590b-801c-4266-b483-035c39c58aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:54.868506 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 921/1821] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:54.872957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e19e7-795b-4351-99f4-9bdd40be4a9b None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:54.882681 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e19e7-795b-4351-99f4-9bdd40be4a9b None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:54.887699 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67aeda-45f7-4c08-88cb-7514fd552d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:54.888306 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67aeda-45f7-4c08-88cb-7514fd552d57 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:54.891738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67aeda-45f7-4c08-88cb-7514fd552d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:54.891738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67aeda-45f7-4c08-88cb-7514fd552d57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:54.891738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67aeda-45f7-4c08-88cb-7514fd552d57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:54.891738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67aeda-45f7-4c08-88cb-7514fd552d57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:54.891738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67aeda-45f7-4c08-88cb-7514fd552d57 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:54.904424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 901/1822] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:34:54.908809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b590b-801c-4266-b483-035c39c58aa8 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:54.919172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b590b-801c-4266-b483-035c39c58aa8 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:54.924482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d7c18-c6bf-4d23-8a4b-d566ccf7f558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:54.925169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d7c18-c6bf-4d23-8a4b-d566ccf7f558 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:54.925523 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d7c18-c6bf-4d23-8a4b-d566ccf7f558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:54.925868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d7c18-c6bf-4d23-8a4b-d566ccf7f558 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:54.926588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d7c18-c6bf-4d23-8a4b-d566ccf7f558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:54.928738 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d7c18-c6bf-4d23-8a4b-d566ccf7f558 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:55.020685 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67aeda-45f7-4c08-88cb-7514fd552d57 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:55.024023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-fe67aeda-45f7-4c08-88cb-7514fd552d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:55.041375 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67aeda-45f7-4c08-88cb-7514fd552d57 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:55.046707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ab72b-caef-48d2-8b84-e9d4c2ce3774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:55.047394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ab72b-caef-48d2-8b84-e9d4c2ce3774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:55.047993 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ab72b-caef-48d2-8b84-e9d4c2ce3774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:55.048120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ab72b-caef-48d2-8b84-e9d4c2ce3774 admin admin] PATH_INFO: `/v3/projects/af5cec233bd443cd9846dbe33e4d76b8` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:55.051164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ab72b-caef-48d2-8b84-e9d4c2ce3774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af5cec233bd443cd9846dbe33e4d76b8)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:55.053175 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ab72b-caef-48d2-8b84-e9d4c2ce3774 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:55.126368 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 922/1823] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:54 2022] POST /identity/v3/projects => generated 373 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:55.127986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d7c18-c6bf-4d23-8a4b-d566ccf7f558 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:55.192195 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d7c18-c6bf-4d23-8a4b-d566ccf7f558 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:55.196938 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093f927-7291-4e4f-82ac-d0f433061d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:55.197626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093f927-7291-4e4f-82ac-d0f433061d89 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:55.197974 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093f927-7291-4e4f-82ac-d0f433061d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:55.198315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093f927-7291-4e4f-82ac-d0f433061d89 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:55.199239 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093f927-7291-4e4f-82ac-d0f433061d89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:55.201566 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093f927-7291-4e4f-82ac-d0f433061d89 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:55.238901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-007ab72b-caef-48d2-8b84-e9d4c2ce3774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af5cec233bd443cd9846dbe33e4d76b8', 'request_id': 'req-007ab72b-caef-48d2-8b84-e9d4c2ce3774'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:55.244155 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-007ab72b-caef-48d2-8b84-e9d4c2ce3774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af5cec233bd443cd9846dbe33e4d76b8', 'request_id': 'req-007ab72b-caef-48d2-8b84-e9d4c2ce3774'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:55.315172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 923/1824] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:55 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:55.319538 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093f927-7291-4e4f-82ac-d0f433061d89 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:55.329279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093f927-7291-4e4f-82ac-d0f433061d89 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:55.342202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-007ab72b-caef-48d2-8b84-e9d4c2ce3774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af5cec233bd443cd9846dbe33e4d76b8', 'request_id': 'req-007ab72b-caef-48d2-8b84-e9d4c2ce3774'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:55.354837 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-007ab72b-caef-48d2-8b84-e9d4c2ce3774 admin admin] The token cache is being invalidate because project af5cec233bd443cd9846dbe33e4d76b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:34:55.355135 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-007ab72b-caef-48d2-8b84-e9d4c2ce3774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:55.355818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-007ab72b-caef-48d2-8b84-e9d4c2ce3774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:34:55.357585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 902/1825] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:34:55 2022] DELETE /identity/v3/projects/af5cec233bd443cd9846dbe33e4d76b8 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:34:55.362169 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f674e1c-d752-436b-9edf-04c4153f2a43 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:55.362463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f674e1c-d752-436b-9edf-04c4153f2a43 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:55.362745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f674e1c-d752-436b-9edf-04c4153f2a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:55.383511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-9f674e1c-d752-436b-9edf-04c4153f2a43 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:55.384748 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9f674e1c-d752-436b-9edf-04c4153f2a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:55.387711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4d158-c548-4268-a12e-b3669094d34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:55.388326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4d158-c548-4268-a12e-b3669094d34a admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:55.388611 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4d158-c548-4268-a12e-b3669094d34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:55.388883 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4d158-c548-4268-a12e-b3669094d34a admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:55.389555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4d158-c548-4268-a12e-b3669094d34a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:55.391249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4d158-c548-4268-a12e-b3669094d34a admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:55.393606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 903/1826] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:34:55.401270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540c8cf-1819-4d0b-890e-714db8483a42 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:55.401555 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540c8cf-1819-4d0b-890e-714db8483a42 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:55.401831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540c8cf-1819-4d0b-890e-714db8483a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:55.401979 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 924/1827] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:34:55.405886 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4d158-c548-4268-a12e-b3669094d34a admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:55.407557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-16e4d158-c548-4268-a12e-b3669094d34a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:55.419937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4d158-c548-4268-a12e-b3669094d34a admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:55.421099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-9540c8cf-1819-4d0b-890e-714db8483a42 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:55.422411 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9540c8cf-1819-4d0b-890e-714db8483a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:55.425145 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97952537-201f-4c82-b417-baf676f8e8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:55.425746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97952537-201f-4c82-b417-baf676f8e8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:55.426031 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97952537-201f-4c82-b417-baf676f8e8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:55.426311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97952537-201f-4c82-b417-baf676f8e8d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:55.426985 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97952537-201f-4c82-b417-baf676f8e8d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:55.428571 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97952537-201f-4c82-b417-baf676f8e8d6 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:55.431560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 904/1828] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:34:55.433143 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:55.435440 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540c8cf-1819-4d0b-890e-714db8483a42 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:55.439962 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 925/1829] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:34:55.442696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97952537-201f-4c82-b417-baf676f8e8d6 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:55.444444 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540c8cf-1819-4d0b-890e-714db8483a42 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:55.449166 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7320d-f2f8-412e-87f0-66da2bff3f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:55.449756 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7320d-f2f8-412e-87f0-66da2bff3f06 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:55.450036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7320d-f2f8-412e-87f0-66da2bff3f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:55.450308 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7320d-f2f8-412e-87f0-66da2bff3f06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:55.450987 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7320d-f2f8-412e-87f0-66da2bff3f06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:34:55.451289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7320d-f2f8-412e-87f0-66da2bff3f06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:55.452726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97952537-201f-4c82-b417-baf676f8e8d6 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:55.452960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7320d-f2f8-412e-87f0-66da2bff3f06 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:55.458554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd166a-c6af-4663-8639-a7ba7bdb27e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:55.459305 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd166a-c6af-4663-8639-a7ba7bdb27e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:55.459589 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd166a-c6af-4663-8639-a7ba7bdb27e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:55.459868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd166a-c6af-4663-8639-a7ba7bdb27e6 admin admin] PATH_INFO: `/v3/projects/f1bf7869124d442a92b98700218497f0/users/34786776aa5b434abfc9f3ee3b5bc1d3/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:55.465759 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 905/1830] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:34:55.471881 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7320d-f2f8-412e-87f0-66da2bff3f06 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:55.475252 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd166a-c6af-4663-8639-a7ba7bdb27e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1bf7869124d442a92b98700218497f0, user_id=34786776aa5b434abfc9f3ee3b5bc1d3, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:55.477897 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd166a-c6af-4663-8639-a7ba7bdb27e6 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:55.481773 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7320d-f2f8-412e-87f0-66da2bff3f06 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:55.487177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c5bb5-ffb6-4629-a3bf-4b84aff7ae3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:55.487834 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c5bb5-ffb6-4629-a3bf-4b84aff7ae3e admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:55.493695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c5bb5-ffb6-4629-a3bf-4b84aff7ae3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:55.493695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c5bb5-ffb6-4629-a3bf-4b84aff7ae3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:55.493695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c5bb5-ffb6-4629-a3bf-4b84aff7ae3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:55.493695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c5bb5-ffb6-4629-a3bf-4b84aff7ae3e admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:55.563254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 926/1831] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:55 2022] PUT /identity/v3/projects/f1bf7869124d442a92b98700218497f0/users/34786776aa5b434abfc9f3ee3b5bc1d3/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:34:55.573534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407c5dd-e960-47ac-91b5-092bb91f35b0 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:55.573901 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407c5dd-e960-47ac-91b5-092bb91f35b0 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:55.574180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407c5dd-e960-47ac-91b5-092bb91f35b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:55.584492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:55.593140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 906/1832] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:55 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:34:55.605419 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-0407c5dd-e960-47ac-91b5-092bb91f35b0 None None] MFA Rules not processed for user `34786776aa5b434abfc9f3ee3b5bc1d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:55.606673 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0407c5dd-e960-47ac-91b5-092bb91f35b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:55.613901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c5bb5-ffb6-4629-a3bf-4b84aff7ae3e admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:55.672040 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 927/1833] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:55.677200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407c5dd-e960-47ac-91b5-092bb91f35b0 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:55.691589 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c5bb5-ffb6-4629-a3bf-4b84aff7ae3e admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:55.697572 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc54b32-c6c5-4abb-9d2e-5fc1cc3cb2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:55.698525 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc54b32-c6c5-4abb-9d2e-5fc1cc3cb2ca admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:55.699010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc54b32-c6c5-4abb-9d2e-5fc1cc3cb2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:55.699455 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc54b32-c6c5-4abb-9d2e-5fc1cc3cb2ca admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:55.700236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc54b32-c6c5-4abb-9d2e-5fc1cc3cb2ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:55.702407 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc54b32-c6c5-4abb-9d2e-5fc1cc3cb2ca admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:55.755602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407c5dd-e960-47ac-91b5-092bb91f35b0 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:55.760263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde25aa-1d51-422c-b76c-5eb33aa539cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:55.760875 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde25aa-1d51-422c-b76c-5eb33aa539cc admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:55.761156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde25aa-1d51-422c-b76c-5eb33aa539cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:55.761432 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde25aa-1d51-422c-b76c-5eb33aa539cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:55.762069 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde25aa-1d51-422c-b76c-5eb33aa539cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:55.763991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde25aa-1d51-422c-b76c-5eb33aa539cc admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:55.803871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 907/1834] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:55 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:34:55.805317 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc54b32-c6c5-4abb-9d2e-5fc1cc3cb2ca admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:55.815756 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc54b32-c6c5-4abb-9d2e-5fc1cc3cb2ca admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:55.820660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ef06c-1c73-443e-9fd7-66ea83c68f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:55.821271 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ef06c-1c73-443e-9fd7-66ea83c68f98 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:55.821557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ef06c-1c73-443e-9fd7-66ea83c68f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:55.821855 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ef06c-1c73-443e-9fd7-66ea83c68f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:55.822578 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ef06c-1c73-443e-9fd7-66ea83c68f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:55.824331 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ef06c-1c73-443e-9fd7-66ea83c68f98 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:55.834729 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 908/1835] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:55.838950 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ef06c-1c73-443e-9fd7-66ea83c68f98 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:55.849543 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ef06c-1c73-443e-9fd7-66ea83c68f98 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:55.854354 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9c19d-eeb8-4a4f-a879-10c2ade6959a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:55.854990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9c19d-eeb8-4a4f-a879-10c2ade6959a admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:55.855277 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9c19d-eeb8-4a4f-a879-10c2ade6959a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:55.855555 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9c19d-eeb8-4a4f-a879-10c2ade6959a admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:55.856189 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9c19d-eeb8-4a4f-a879-10c2ade6959a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:55.857791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9c19d-eeb8-4a4f-a879-10c2ade6959a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:55.868355 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 909/1836] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:34:55.872432 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9c19d-eeb8-4a4f-a879-10c2ade6959a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:55.883459 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9c19d-eeb8-4a4f-a879-10c2ade6959a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:55.889067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d05b27-76c9-47da-9fca-9008ae8c871e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:55.889777 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d05b27-76c9-47da-9fca-9008ae8c871e admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:55.890045 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d05b27-76c9-47da-9fca-9008ae8c871e admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:55.890284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d05b27-76c9-47da-9fca-9008ae8c871e admin admin] PATH_INFO: `/v3/projects/3c30822e89fe4cc7a444dee7c417c511/users/6fdd0b30046e448da0f16f4aafd8d1bd/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:55.906078 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d05b27-76c9-47da-9fca-9008ae8c871e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c30822e89fe4cc7a444dee7c417c511, user_id=6fdd0b30046e448da0f16f4aafd8d1bd, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:55.909074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d05b27-76c9-47da-9fca-9008ae8c871e admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:55.913543 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 928/1837] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:55 2022] POST /identity/v3/projects => generated 375 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:34:55.916803 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde25aa-1d51-422c-b76c-5eb33aa539cc admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:55.978258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde25aa-1d51-422c-b76c-5eb33aa539cc admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:55.983050 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be40934-f2aa-49e9-b6cb-ee08ba99bcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:55.983657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be40934-f2aa-49e9-b6cb-ee08ba99bcad admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:55.983937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be40934-f2aa-49e9-b6cb-ee08ba99bcad admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:55.984211 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be40934-f2aa-49e9-b6cb-ee08ba99bcad admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:55.984836 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be40934-f2aa-49e9-b6cb-ee08ba99bcad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:55.986808 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be40934-f2aa-49e9-b6cb-ee08ba99bcad admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:57.359077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 910/1838] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:55 2022] PUT /identity/v3/projects/3c30822e89fe4cc7a444dee7c417c511/users/6fdd0b30046e448da0f16f4aafd8d1bd/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 1487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:34:57.371156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43337726-10e0-493d-ac9a-e695d3ba456b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:57.371497 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43337726-10e0-493d-ac9a-e695d3ba456b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:57.371776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43337726-10e0-493d-ac9a-e695d3ba456b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:57.402949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-43337726-10e0-493d-ac9a-e695d3ba456b None None] MFA Rules not processed for user `6fdd0b30046e448da0f16f4aafd8d1bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:57.404224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-43337726-10e0-493d-ac9a-e695d3ba456b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:57.479002 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 911/1839] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:57.480020 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43337726-10e0-493d-ac9a-e695d3ba456b None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:57.537212 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 929/1840] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:55 2022] POST /identity/v3/users => generated 294 bytes in 1621 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:34:57.541658 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be40934-f2aa-49e9-b6cb-ee08ba99bcad admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:57.552764 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be40934-f2aa-49e9-b6cb-ee08ba99bcad admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:57.557609 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c8c26-d53e-4f83-8f41-1490c4fc8b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:57.558221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43337726-10e0-493d-ac9a-e695d3ba456b None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:57.558343 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c8c26-d53e-4f83-8f41-1490c4fc8b22 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:57.558501 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c8c26-d53e-4f83-8f41-1490c4fc8b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:57.558782 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c8c26-d53e-4f83-8f41-1490c4fc8b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:57.559461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c8c26-d53e-4f83-8f41-1490c4fc8b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:57.561078 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c8c26-d53e-4f83-8f41-1490c4fc8b22 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:57.565393 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007c1bd-4db9-4ac7-ae64-f4049e13df93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:57.567139 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007c1bd-4db9-4ac7-ae64-f4049e13df93 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:57.567139 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007c1bd-4db9-4ac7-ae64-f4049e13df93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:57.567139 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007c1bd-4db9-4ac7-ae64-f4049e13df93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:57.568149 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007c1bd-4db9-4ac7-ae64-f4049e13df93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:57.571933 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007c1bd-4db9-4ac7-ae64-f4049e13df93 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:57.587159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 930/1841] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:57.595952 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c8c26-d53e-4f83-8f41-1490c4fc8b22 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:57.610131 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c8c26-d53e-4f83-8f41-1490c4fc8b22 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:57.617789 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a75161-2288-41d0-bae0-ddac44c5fef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:57.617789 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a75161-2288-41d0-bae0-ddac44c5fef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:57.617789 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a75161-2288-41d0-bae0-ddac44c5fef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:57.617789 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a75161-2288-41d0-bae0-ddac44c5fef3 admin admin] PATH_INFO: `/v3/projects/ed7dd5aa5a8f468083d882670a2c7c2b/users/51109d9810eb4e3aa320e134d31a38c5/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:57.636097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a75161-2288-41d0-bae0-ddac44c5fef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7dd5aa5a8f468083d882670a2c7c2b, user_id=51109d9810eb4e3aa320e134d31a38c5, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:57.638926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a75161-2288-41d0-bae0-ddac44c5fef3 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:57.682760 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 912/1842] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:34:57 2022] POST /identity/v3/projects => generated 377 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:57.686776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007c1bd-4db9-4ac7-ae64-f4049e13df93 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:57.751609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007c1bd-4db9-4ac7-ae64-f4049e13df93 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:57.756961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc605f6-2acc-4452-8639-8b04b22b4553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:57.757573 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc605f6-2acc-4452-8639-8b04b22b4553 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:57.757861 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc605f6-2acc-4452-8639-8b04b22b4553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:57.758141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc605f6-2acc-4452-8639-8b04b22b4553 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:57.758803 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc605f6-2acc-4452-8639-8b04b22b4553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:57.760738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc605f6-2acc-4452-8639-8b04b22b4553 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:57.773891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 931/1843] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:57 2022] PUT /identity/v3/projects/ed7dd5aa5a8f468083d882670a2c7c2b/users/51109d9810eb4e3aa320e134d31a38c5/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:34:57.777945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a75161-2288-41d0-bae0-ddac44c5fef3 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:57.840106 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a75161-2288-41d0-bae0-ddac44c5fef3 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:57.844831 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713c4fc-e5de-4723-b64f-e8a2dbf7cbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:57.845431 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713c4fc-e5de-4723-b64f-e8a2dbf7cbb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:57.845715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713c4fc-e5de-4723-b64f-e8a2dbf7cbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:57.846034 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713c4fc-e5de-4723-b64f-e8a2dbf7cbb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:57.846719 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713c4fc-e5de-4723-b64f-e8a2dbf7cbb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:57.848349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713c4fc-e5de-4723-b64f-e8a2dbf7cbb7 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:57.857840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 932/1844] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:34:57.862562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713c4fc-e5de-4723-b64f-e8a2dbf7cbb7 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:57.871978 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713c4fc-e5de-4723-b64f-e8a2dbf7cbb7 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:57.876521 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da96930-10f8-44ca-af91-5f9f48032eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:57.877830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da96930-10f8-44ca-af91-5f9f48032eda admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:57.878119 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da96930-10f8-44ca-af91-5f9f48032eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:57.878397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da96930-10f8-44ca-af91-5f9f48032eda admin admin] PATH_INFO: `/v3/projects/ed7dd5aa5a8f468083d882670a2c7c2b/users/51109d9810eb4e3aa320e134d31a38c5/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:57.880438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da96930-10f8-44ca-af91-5f9f48032eda admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7dd5aa5a8f468083d882670a2c7c2b, user_id=51109d9810eb4e3aa320e134d31a38c5, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:57.883062 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da96930-10f8-44ca-af91-5f9f48032eda admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:57.885407 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 913/1845] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:34:57 2022] POST /identity/v3/users => generated 295 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:34:57.889143 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc605f6-2acc-4452-8639-8b04b22b4553 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:57.893537 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 933/1846] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:57 2022] PUT /identity/v3/projects/ed7dd5aa5a8f468083d882670a2c7c2b/users/51109d9810eb4e3aa320e134d31a38c5/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:34:57.904479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731c31b-c7dc-4510-8efa-5f32adcba4d1 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:57.904769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731c31b-c7dc-4510-8efa-5f32adcba4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:57.905045 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731c31b-c7dc-4510-8efa-5f32adcba4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:57.907860 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc605f6-2acc-4452-8639-8b04b22b4553 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:57.935737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-a731c31b-c7dc-4510-8efa-5f32adcba4d1 None None] MFA Rules not processed for user `51109d9810eb4e3aa320e134d31a38c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:57.937011 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a731c31b-c7dc-4510-8efa-5f32adcba4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:57.969233 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c5703-9dc6-4347-bef6-4082c353d516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:57.969921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c5703-9dc6-4347-bef6-4082c353d516 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:57.970448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c5703-9dc6-4347-bef6-4082c353d516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:57.970627 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c5703-9dc6-4347-bef6-4082c353d516 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:57.971301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c5703-9dc6-4347-bef6-4082c353d516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:57.972886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c5703-9dc6-4347-bef6-4082c353d516 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:57.983492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 914/1847] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:34:57.987878 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c5703-9dc6-4347-bef6-4082c353d516 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:57.997697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c5703-9dc6-4347-bef6-4082c353d516 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:58.002439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d5843-fcb3-4439-a5b0-672904b1a1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:58.003113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d5843-fcb3-4439-a5b0-672904b1a1d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:58.003394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d5843-fcb3-4439-a5b0-672904b1a1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:58.003668 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d5843-fcb3-4439-a5b0-672904b1a1d5 admin admin] PATH_INFO: `/v3/projects/cbeac48c23bf40cf9a7132405f45b806/users/d0713340ad8045afb5c9b2c8427aeeda/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:58.011496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 934/1848] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:34:58.018022 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d5843-fcb3-4439-a5b0-672904b1a1d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbeac48c23bf40cf9a7132405f45b806, user_id=d0713340ad8045afb5c9b2c8427aeeda, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:58.020678 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d5843-fcb3-4439-a5b0-672904b1a1d5 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:58.025510 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731c31b-c7dc-4510-8efa-5f32adcba4d1 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:58.026911 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a731c31b-c7dc-4510-8efa-5f32adcba4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:58.082921 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731c31b-c7dc-4510-8efa-5f32adcba4d1 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:58.089178 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 915/1849] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:57 2022] PUT /identity/v3/projects/cbeac48c23bf40cf9a7132405f45b806/users/d0713340ad8045afb5c9b2c8427aeeda/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:34:58.094248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d5843-fcb3-4439-a5b0-672904b1a1d5 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:58.136334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1510160-9bb0-45a7-b4d1-aebeac837497 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:58.136943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1510160-9bb0-45a7-b4d1-aebeac837497 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:58.137328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1510160-9bb0-45a7-b4d1-aebeac837497 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:58.137530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1510160-9bb0-45a7-b4d1-aebeac837497 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:58.156987 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d5843-fcb3-4439-a5b0-672904b1a1d5 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:58.164461 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d59a50-6988-48ce-ac45-053ecbbd9f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:58.165098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d59a50-6988-48ce-ac45-053ecbbd9f81 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:58.165387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d59a50-6988-48ce-ac45-053ecbbd9f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:58.165648 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d59a50-6988-48ce-ac45-053ecbbd9f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:58.166304 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d59a50-6988-48ce-ac45-053ecbbd9f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:58.168038 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d59a50-6988-48ce-ac45-053ecbbd9f81 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:58.179713 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 916/1850] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:34:58.184774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d59a50-6988-48ce-ac45-053ecbbd9f81 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:58.195832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d59a50-6988-48ce-ac45-053ecbbd9f81 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:58.202009 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6f091-0d65-414d-ab3c-6d7ea353931e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:58.204846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6f091-0d65-414d-ab3c-6d7ea353931e admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:58.204846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6f091-0d65-414d-ab3c-6d7ea353931e admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:58.204846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6f091-0d65-414d-ab3c-6d7ea353931e admin admin] PATH_INFO: `/v3/projects/cbeac48c23bf40cf9a7132405f45b806/users/d0713340ad8045afb5c9b2c8427aeeda/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:58.205133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1510160-9bb0-45a7-b4d1-aebeac837497 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:58.206053 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6f091-0d65-414d-ab3c-6d7ea353931e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbeac48c23bf40cf9a7132405f45b806, user_id=d0713340ad8045afb5c9b2c8427aeeda, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:58.206427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1510160-9bb0-45a7-b4d1-aebeac837497 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:58.209425 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6f091-0d65-414d-ab3c-6d7ea353931e admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:58.217944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 935/1851] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:34:58.227948 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 917/1852] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:34:58 2022] PUT /identity/v3/projects/cbeac48c23bf40cf9a7132405f45b806/users/d0713340ad8045afb5c9b2c8427aeeda/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:34:58.233917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224e24c-1cbf-483c-b01b-ea4f859f9718 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:58.234262 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224e24c-1cbf-483c-b01b-ea4f859f9718 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:58.234647 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224e24c-1cbf-483c-b01b-ea4f859f9718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:58.273910 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-a224e24c-1cbf-483c-b01b-ea4f859f9718 None None] MFA Rules not processed for user `d0713340ad8045afb5c9b2c8427aeeda`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:34:58.275429 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a224e24c-1cbf-483c-b01b-ea4f859f9718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:58.350110 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 936/1853] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:34:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:34:58.400085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:34:58.465300 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6f091-0d65-414d-ab3c-6d7ea353931e admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:34:58.467689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-83b6f091-0d65-414d-ab3c-6d7ea353931e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:34:58.531433 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6f091-0d65-414d-ab3c-6d7ea353931e admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:34:58.535102 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a547d-cdf0-4bf1-a04e-4120e788f76d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:34:58.535716 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a547d-cdf0-4bf1-a04e-4120e788f76d service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:34:58.535997 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a547d-cdf0-4bf1-a04e-4120e788f76d service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:34:58.536269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a547d-cdf0-4bf1-a04e-4120e788f76d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:34:58.591878 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a547d-cdf0-4bf1-a04e-4120e788f76d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:34:58.593381 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a547d-cdf0-4bf1-a04e-4120e788f76d service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:34:58.602569 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 918/1854] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:34:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:34:58.671340 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:00.085665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f3ae5-c76a-4616-99ac-48bfaf447419 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:00.086295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f3ae5-c76a-4616-99ac-48bfaf447419 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:00.086295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f3ae5-c76a-4616-99ac-48bfaf447419 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:00.101801 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 937/1855] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 5 14:35:00 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:35:00.155094 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543cc3a-bf34-47d8-b3fa-f2f9f81a4159 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:00.155094 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543cc3a-bf34-47d8-b3fa-f2f9f81a4159 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:00.155094 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543cc3a-bf34-47d8-b3fa-f2f9f81a4159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:00.173197 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-8543cc3a-bf34-47d8-b3fa-f2f9f81a4159 None None] MFA Rules not processed for user `709b9ae82dcd47408fc9a6d54a54428a`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:35:00.175007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8543cc3a-bf34-47d8-b3fa-f2f9f81a4159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:00.246298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 919/1856] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 5 14:35:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:35:00.268011 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f3ae5-c76a-4616-99ac-48bfaf447419 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:00.269956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ae3f3ae5-c76a-4616-99ac-48bfaf447419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:00.289556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f3ae5-c76a-4616-99ac-48bfaf447419 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:00.295789 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a704a-736b-4626-ba8d-3f2a9d6ba324 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:00.296474 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a704a-736b-4626-ba8d-3f2a9d6ba324 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:00.296761 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a704a-736b-4626-ba8d-3f2a9d6ba324 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:00.297051 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a704a-736b-4626-ba8d-3f2a9d6ba324 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:00.307120 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a704a-736b-4626-ba8d-3f2a9d6ba324 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:00.308767 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a704a-736b-4626-ba8d-3f2a9d6ba324 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:00.320660 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 938/1857] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:35:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:35:00.323073 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:00.497402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:02.148611 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:02.680519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543cc3a-bf34-47d8-b3fa-f2f9f81a4159 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:02.742458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543cc3a-bf34-47d8-b3fa-f2f9f81a4159 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:02.747246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74cdd9-f4cb-4012-a5f5-87d498e1ad32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:02.747964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74cdd9-f4cb-4012-a5f5-87d498e1ad32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:02.748192 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74cdd9-f4cb-4012-a5f5-87d498e1ad32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:02.748471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74cdd9-f4cb-4012-a5f5-87d498e1ad32 admin admin] PATH_INFO: `/v3/users/34786776aa5b434abfc9f3ee3b5bc1d3` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:02.749598 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74cdd9-f4cb-4012-a5f5-87d498e1ad32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34786776aa5b434abfc9f3ee3b5bc1d3)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:02.751603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74cdd9-f4cb-4012-a5f5-87d498e1ad32 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:02.822777 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:02.923799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-cf74cdd9-f4cb-4012-a5f5-87d498e1ad32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34786776aa5b434abfc9f3ee3b5bc1d3'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:03.022516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-cf74cdd9-f4cb-4012-a5f5-87d498e1ad32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34786776aa5b434abfc9f3ee3b5bc1d3'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:03.023167 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-cf74cdd9-f4cb-4012-a5f5-87d498e1ad32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34786776aa5b434abfc9f3ee3b5bc1d3'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:03.036627 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-cf74cdd9-f4cb-4012-a5f5-87d498e1ad32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34786776aa5b434abfc9f3ee3b5bc1d3'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:03.043513 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-cf74cdd9-f4cb-4012-a5f5-87d498e1ad32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34786776aa5b434abfc9f3ee3b5bc1d3'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:03.049769 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 920/1858] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:35:02 2022] DELETE /identity/v3/users/34786776aa5b434abfc9f3ee3b5bc1d3 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:35:03.054503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3382a-89d4-4c4e-b8bb-12e2789b4476 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:03.054806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3382a-89d4-4c4e-b8bb-12e2789b4476 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:03.055148 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3382a-89d4-4c4e-b8bb-12e2789b4476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:03.073692 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-82b3382a-89d4-4c4e-b8bb-12e2789b4476 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:35:03.074932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-82b3382a-89d4-4c4e-b8bb-12e2789b4476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:03.132811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 939/1859] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:35:03.143107 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74cdd9-f4cb-4012-a5f5-87d498e1ad32 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:03.144541 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-cf74cdd9-f4cb-4012-a5f5-87d498e1ad32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:03.199924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74cdd9-f4cb-4012-a5f5-87d498e1ad32 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:03.203436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5a2cd-5358-4621-94bd-e552a2c6c0b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:03.204042 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5a2cd-5358-4621-94bd-e552a2c6c0b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:03.204317 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5a2cd-5358-4621-94bd-e552a2c6c0b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:03.204588 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5a2cd-5358-4621-94bd-e552a2c6c0b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:03.213946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5a2cd-5358-4621-94bd-e552a2c6c0b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:03.215509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5a2cd-5358-4621-94bd-e552a2c6c0b0 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:03.225370 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 921/1860] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:35:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:35:04.063930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3382a-89d4-4c4e-b8bb-12e2789b4476 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:04.068708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-82b3382a-89d4-4c4e-b8bb-12e2789b4476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:04.080277 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3382a-89d4-4c4e-b8bb-12e2789b4476 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:04.084867 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7333c01-c055-4311-b86a-ba2e42e9ccb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:04.085503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7333c01-c055-4311-b86a-ba2e42e9ccb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:04.085785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7333c01-c055-4311-b86a-ba2e42e9ccb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:04.086054 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7333c01-c055-4311-b86a-ba2e42e9ccb1 admin admin] PATH_INFO: `/v3/projects/f1bf7869124d442a92b98700218497f0` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:04.088239 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7333c01-c055-4311-b86a-ba2e42e9ccb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1bf7869124d442a92b98700218497f0)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:04.090746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7333c01-c055-4311-b86a-ba2e42e9ccb1 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:04.205022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-c7333c01-c055-4311-b86a-ba2e42e9ccb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1bf7869124d442a92b98700218497f0', 'request_id': 'req-c7333c01-c055-4311-b86a-ba2e42e9ccb1'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:04.211170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-c7333c01-c055-4311-b86a-ba2e42e9ccb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1bf7869124d442a92b98700218497f0', 'request_id': 'req-c7333c01-c055-4311-b86a-ba2e42e9ccb1'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:04.230608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:04.327668 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-c7333c01-c055-4311-b86a-ba2e42e9ccb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1bf7869124d442a92b98700218497f0', 'request_id': 'req-c7333c01-c055-4311-b86a-ba2e42e9ccb1'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:04.339147 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-c7333c01-c055-4311-b86a-ba2e42e9ccb1 admin admin] The token cache is being invalidate because project f1bf7869124d442a92b98700218497f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:35:04.339448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-c7333c01-c055-4311-b86a-ba2e42e9ccb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:04.340749 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-c7333c01-c055-4311-b86a-ba2e42e9ccb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:04.342287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 940/1861] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:35:04 2022] DELETE /identity/v3/projects/f1bf7869124d442a92b98700218497f0 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:35:04.346933 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5a2cd-5358-4621-94bd-e552a2c6c0b0 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:04.356633 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5a2cd-5358-4621-94bd-e552a2c6c0b0 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:04.417339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8447e-f242-48c7-80fa-aad727c9a941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:04.418084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8447e-f242-48c7-80fa-aad727c9a941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:04.418370 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8447e-f242-48c7-80fa-aad727c9a941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:04.418639 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8447e-f242-48c7-80fa-aad727c9a941 admin admin] PATH_INFO: `/v3/users/51109d9810eb4e3aa320e134d31a38c5` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:04.419804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8447e-f242-48c7-80fa-aad727c9a941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51109d9810eb4e3aa320e134d31a38c5)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:04.421703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8447e-f242-48c7-80fa-aad727c9a941 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:04.578739 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-38f8447e-f242-48c7-80fa-aad727c9a941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51109d9810eb4e3aa320e134d31a38c5'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:04.705474 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-38f8447e-f242-48c7-80fa-aad727c9a941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51109d9810eb4e3aa320e134d31a38c5'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:04.706257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-38f8447e-f242-48c7-80fa-aad727c9a941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51109d9810eb4e3aa320e134d31a38c5'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:04.720018 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-38f8447e-f242-48c7-80fa-aad727c9a941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51109d9810eb4e3aa320e134d31a38c5'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:04.733467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-38f8447e-f242-48c7-80fa-aad727c9a941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51109d9810eb4e3aa320e134d31a38c5'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:04.739980 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 922/1862] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:35:04 2022] DELETE /identity/v3/users/51109d9810eb4e3aa320e134d31a38c5 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:35:04.794380 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:05.197021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7333c01-c055-4311-b86a-ba2e42e9ccb1 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:05.197021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c7333c01-c055-4311-b86a-ba2e42e9ccb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:05.211472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7333c01-c055-4311-b86a-ba2e42e9ccb1 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:05.268330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552ac8a-d769-4fa4-a0db-e60a85c4551c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:05.268964 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552ac8a-d769-4fa4-a0db-e60a85c4551c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:05.269243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552ac8a-d769-4fa4-a0db-e60a85c4551c admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:05.269517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552ac8a-d769-4fa4-a0db-e60a85c4551c admin admin] PATH_INFO: `/v3/projects/ed7dd5aa5a8f468083d882670a2c7c2b` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:05.270608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552ac8a-d769-4fa4-a0db-e60a85c4551c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed7dd5aa5a8f468083d882670a2c7c2b)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:05.272580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552ac8a-d769-4fa4-a0db-e60a85c4551c admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:05.405196 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-2552ac8a-d769-4fa4-a0db-e60a85c4551c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed7dd5aa5a8f468083d882670a2c7c2b', 'request_id': 'req-2552ac8a-d769-4fa4-a0db-e60a85c4551c'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:05.412517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-2552ac8a-d769-4fa4-a0db-e60a85c4551c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed7dd5aa5a8f468083d882670a2c7c2b', 'request_id': 'req-2552ac8a-d769-4fa4-a0db-e60a85c4551c'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:05.503139 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-2552ac8a-d769-4fa4-a0db-e60a85c4551c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed7dd5aa5a8f468083d882670a2c7c2b', 'request_id': 'req-2552ac8a-d769-4fa4-a0db-e60a85c4551c'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:05.514053 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-2552ac8a-d769-4fa4-a0db-e60a85c4551c admin admin] The token cache is being invalidate because project ed7dd5aa5a8f468083d882670a2c7c2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:35:05.514361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-2552ac8a-d769-4fa4-a0db-e60a85c4551c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:05.515175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-2552ac8a-d769-4fa4-a0db-e60a85c4551c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:05.516609 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 941/1863] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:35:05 2022] DELETE /identity/v3/projects/ed7dd5aa5a8f468083d882670a2c7c2b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:35:05.521395 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98abb93-3165-4f38-8e91-735a28480bd1 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:05.521848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98abb93-3165-4f38-8e91-735a28480bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:05.521956 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98abb93-3165-4f38-8e91-735a28480bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:05.542163 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-f98abb93-3165-4f38-8e91-735a28480bd1 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:35:05.543099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f98abb93-3165-4f38-8e91-735a28480bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:05.602289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 923/1864] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:35:05.611762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06a075-1473-4c03-a7b1-55d5ac33134c None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:05.612078 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06a075-1473-4c03-a7b1-55d5ac33134c None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:05.612357 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06a075-1473-4c03-a7b1-55d5ac33134c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:05.631684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-5e06a075-1473-4c03-a7b1-55d5ac33134c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:35:05.632917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5e06a075-1473-4c03-a7b1-55d5ac33134c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:05.640810 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 942/1865] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:35:05.645119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98abb93-3165-4f38-8e91-735a28480bd1 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:05.654841 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98abb93-3165-4f38-8e91-735a28480bd1 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:05.660286 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d2a14-a7ae-4607-a019-a8c4f028faba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:05.660849 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d2a14-a7ae-4607-a019-a8c4f028faba admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:05.661134 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d2a14-a7ae-4607-a019-a8c4f028faba admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:05.661444 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d2a14-a7ae-4607-a019-a8c4f028faba admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:05.665873 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d2a14-a7ae-4607-a019-a8c4f028faba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:35:05.665873 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d2a14-a7ae-4607-a019-a8c4f028faba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:05.665873 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d2a14-a7ae-4607-a019-a8c4f028faba admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:05.675943 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 924/1866] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:35:05.680333 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06a075-1473-4c03-a7b1-55d5ac33134c None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:05.689476 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06a075-1473-4c03-a7b1-55d5ac33134c None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:05.694054 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5819a1-3d52-403c-bf75-2cc55290c198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:05.694648 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5819a1-3d52-403c-bf75-2cc55290c198 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:05.694970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5819a1-3d52-403c-bf75-2cc55290c198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:05.697890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5819a1-3d52-403c-bf75-2cc55290c198 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:05.697890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5819a1-3d52-403c-bf75-2cc55290c198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:05.697890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5819a1-3d52-403c-bf75-2cc55290c198 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:05.801459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 943/1867] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:35:05 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:35:05.805464 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d2a14-a7ae-4607-a019-a8c4f028faba admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:05.815472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d2a14-a7ae-4607-a019-a8c4f028faba admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:05.870907 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57538d2f-a458-44e3-97df-bb6bb2cc8d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:05.871514 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57538d2f-a458-44e3-97df-bb6bb2cc8d1e admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:05.871797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57538d2f-a458-44e3-97df-bb6bb2cc8d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:05.872092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57538d2f-a458-44e3-97df-bb6bb2cc8d1e admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:05.872701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57538d2f-a458-44e3-97df-bb6bb2cc8d1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:05.874620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57538d2f-a458-44e3-97df-bb6bb2cc8d1e admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:05.983132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 925/1868] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:35:05 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:35:05.984409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5819a1-3d52-403c-bf75-2cc55290c198 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:05.995228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5819a1-3d52-403c-bf75-2cc55290c198 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:06.000011 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a250d-0b29-41a4-9f7f-f1f2ddbe8787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:06.000624 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a250d-0b29-41a4-9f7f-f1f2ddbe8787 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:06.000906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a250d-0b29-41a4-9f7f-f1f2ddbe8787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:06.001183 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a250d-0b29-41a4-9f7f-f1f2ddbe8787 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:06.001829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a250d-0b29-41a4-9f7f-f1f2ddbe8787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:06.003517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a250d-0b29-41a4-9f7f-f1f2ddbe8787 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:06.014012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 944/1869] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:35:06.018899 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57538d2f-a458-44e3-97df-bb6bb2cc8d1e admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:06.030566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57538d2f-a458-44e3-97df-bb6bb2cc8d1e admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:06.053292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ef4a1-fb79-41cb-b6ac-5caa3c4f6002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:06.053292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef4a1-fb79-41cb-b6ac-5caa3c4f6002 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:06.053292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef4a1-fb79-41cb-b6ac-5caa3c4f6002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:06.053292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef4a1-fb79-41cb-b6ac-5caa3c4f6002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:06.054257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ef4a1-fb79-41cb-b6ac-5caa3c4f6002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:06.054257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ef4a1-fb79-41cb-b6ac-5caa3c4f6002 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:06.054257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 926/1870] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:35:06.056172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a250d-0b29-41a4-9f7f-f1f2ddbe8787 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:06.066378 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a250d-0b29-41a4-9f7f-f1f2ddbe8787 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:06.071191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980a6cb-9a1e-4465-b93a-9a9d0db09b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:06.071945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980a6cb-9a1e-4465-b93a-9a9d0db09b9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:06.072385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980a6cb-9a1e-4465-b93a-9a9d0db09b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:06.072691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980a6cb-9a1e-4465-b93a-9a9d0db09b9e admin admin] PATH_INFO: `/v3/projects/1d655e61a6b045b38a636aa9b06d3365/users/3ff06d0b603242f990e59ae898a409dd/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:06.093963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980a6cb-9a1e-4465-b93a-9a9d0db09b9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d655e61a6b045b38a636aa9b06d3365, user_id=3ff06d0b603242f990e59ae898a409dd, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:06.096986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980a6cb-9a1e-4465-b93a-9a9d0db09b9e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:06.191450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 945/1871] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:35:06 2022] PUT /identity/v3/projects/1d655e61a6b045b38a636aa9b06d3365/users/3ff06d0b603242f990e59ae898a409dd/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:35:06.201919 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d34c65-0496-4b44-9520-28978745bbfc None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:06.202212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d34c65-0496-4b44-9520-28978745bbfc None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:06.202495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d34c65-0496-4b44-9520-28978745bbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:06.208190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:06.233512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-d8d34c65-0496-4b44-9520-28978745bbfc None None] MFA Rules not processed for user `3ff06d0b603242f990e59ae898a409dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:35:06.234927 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d8d34c65-0496-4b44-9520-28978745bbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:06.309322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 927/1872] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:35:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:35:06.320902 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980a6cb-9a1e-4465-b93a-9a9d0db09b9e admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:06.322636 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7980a6cb-9a1e-4465-b93a-9a9d0db09b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:06.385566 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980a6cb-9a1e-4465-b93a-9a9d0db09b9e admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:06.389392 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48477c-d33f-4031-930b-ecf47d826472 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:06.390005 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48477c-d33f-4031-930b-ecf47d826472 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:06.390284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48477c-d33f-4031-930b-ecf47d826472 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:06.390550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48477c-d33f-4031-930b-ecf47d826472 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:06.399438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48477c-d33f-4031-930b-ecf47d826472 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:06.400904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48477c-d33f-4031-930b-ecf47d826472 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:06.411327 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 946/1873] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:35:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:35:06.551046 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:07.559150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d34c65-0496-4b44-9520-28978745bbfc None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:07.560913 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d8d34c65-0496-4b44-9520-28978745bbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:07.585787 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d34c65-0496-4b44-9520-28978745bbfc None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:07.589624 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ac027-f5e1-4ccd-8208-332dc963d58b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:07.590523 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ac027-f5e1-4ccd-8208-332dc963d58b service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:07.590823 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ac027-f5e1-4ccd-8208-332dc963d58b service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:07.591112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ac027-f5e1-4ccd-8208-332dc963d58b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:07.595446 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-4a3ac027-f5e1-4ccd-8208-332dc963d58b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:07.680890 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ac027-f5e1-4ccd-8208-332dc963d58b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:07.682355 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ac027-f5e1-4ccd-8208-332dc963d58b service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:07.694924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 928/1874] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:35:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:35:07.911057 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:08.482195 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:08.853742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:08.956493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:09.111793 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:09.153980 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:09.207686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:09.242022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:09.321822 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:09.430553 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:19.533896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:19.610706 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=58 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:44.322502 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e3ec5-3c31-441e-b6f8-44a60af7c69b None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:44.323190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e3ec5-3c31-441e-b6f8-44a60af7c69b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:44.323190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e3ec5-3c31-441e-b6f8-44a60af7c69b None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:44.324423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 947/1875] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 5 14:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:35:44.356916 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a07372-084c-4033-bfbb-9d9e4d4d1ce6 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:44.357531 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a07372-084c-4033-bfbb-9d9e4d4d1ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:44.357531 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a07372-084c-4033-bfbb-9d9e4d4d1ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:44.381307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-23a07372-084c-4033-bfbb-9d9e4d4d1ce6 None None] MFA Rules not processed for user `267825dfb0324be0910203cd5abab377`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:35:44.382680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-23a07372-084c-4033-bfbb-9d9e4d4d1ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:44.453159 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 929/1876] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 5 14:35:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:35:44.468833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e3ec5-3c31-441e-b6f8-44a60af7c69b None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:44.471132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-6f1e3ec5-3c31-441e-b6f8-44a60af7c69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:44.493666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e3ec5-3c31-441e-b6f8-44a60af7c69b None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:44.499721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a60925-50db-4c89-b9bb-e56529039006 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:44.500587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a60925-50db-4c89-b9bb-e56529039006 service placement] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:44.501084 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a60925-50db-4c89-b9bb-e56529039006 service placement] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:44.502012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a60925-50db-4c89-b9bb-e56529039006 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:44.513461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a60925-50db-4c89-b9bb-e56529039006 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:44.515080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a60925-50db-4c89-b9bb-e56529039006 service placement] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:44.531643 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 948/1877] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:35:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:35:45.263347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a07372-084c-4033-bfbb-9d9e4d4d1ce6 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:45.264958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-23a07372-084c-4033-bfbb-9d9e4d4d1ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:45.335112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a07372-084c-4033-bfbb-9d9e4d4d1ce6 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:45.341244 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c22187-3d11-4094-b202-7655774c0241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:45.341884 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c22187-3d11-4094-b202-7655774c0241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:45.342165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c22187-3d11-4094-b202-7655774c0241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:45.342441 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c22187-3d11-4094-b202-7655774c0241 admin admin] PATH_INFO: `/v3/users/71db141bb2f14a0bbe9a750427e54442` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:45.343714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c22187-3d11-4094-b202-7655774c0241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71db141bb2f14a0bbe9a750427e54442)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:45.345836 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c22187-3d11-4094-b202-7655774c0241 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:45.554365 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-86c22187-3d11-4094-b202-7655774c0241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71db141bb2f14a0bbe9a750427e54442'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:45.613114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-86c22187-3d11-4094-b202-7655774c0241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71db141bb2f14a0bbe9a750427e54442'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:45.614117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-86c22187-3d11-4094-b202-7655774c0241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71db141bb2f14a0bbe9a750427e54442'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:45.633247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-86c22187-3d11-4094-b202-7655774c0241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71db141bb2f14a0bbe9a750427e54442'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:45.643495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-86c22187-3d11-4094-b202-7655774c0241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71db141bb2f14a0bbe9a750427e54442'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:45.653109 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 930/1878] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:35:45 2022] DELETE /identity/v3/users/71db141bb2f14a0bbe9a750427e54442 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:35:45.941413 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a60925-50db-4c89-b9bb-e56529039006 service placement] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:45.952807 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a60925-50db-4c89-b9bb-e56529039006 service placement] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:46.018809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba1636-416f-4d3f-a06e-b26b7f8bf551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:46.019692 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba1636-416f-4d3f-a06e-b26b7f8bf551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:46.019979 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba1636-416f-4d3f-a06e-b26b7f8bf551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:46.020259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba1636-416f-4d3f-a06e-b26b7f8bf551 admin admin] PATH_INFO: `/v3/projects/1e10bc2823424ea6acfc0fc82a3890dd` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:46.021769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba1636-416f-4d3f-a06e-b26b7f8bf551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e10bc2823424ea6acfc0fc82a3890dd)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:46.023818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba1636-416f-4d3f-a06e-b26b7f8bf551 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:46.154773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-34ba1636-416f-4d3f-a06e-b26b7f8bf551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e10bc2823424ea6acfc0fc82a3890dd', 'request_id': 'req-34ba1636-416f-4d3f-a06e-b26b7f8bf551'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:46.161133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-34ba1636-416f-4d3f-a06e-b26b7f8bf551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e10bc2823424ea6acfc0fc82a3890dd', 'request_id': 'req-34ba1636-416f-4d3f-a06e-b26b7f8bf551'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:46.216590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-34ba1636-416f-4d3f-a06e-b26b7f8bf551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e10bc2823424ea6acfc0fc82a3890dd', 'request_id': 'req-34ba1636-416f-4d3f-a06e-b26b7f8bf551'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:46.229503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-34ba1636-416f-4d3f-a06e-b26b7f8bf551 admin admin] The token cache is being invalidate because project 1e10bc2823424ea6acfc0fc82a3890dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:35:46.229932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-34ba1636-416f-4d3f-a06e-b26b7f8bf551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:46.230635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-34ba1636-416f-4d3f-a06e-b26b7f8bf551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:46.232543 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 949/1879] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:35:45 2022] DELETE /identity/v3/projects/1e10bc2823424ea6acfc0fc82a3890dd => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:35:46.236297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c22187-3d11-4094-b202-7655774c0241 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:46.237707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-86c22187-3d11-4094-b202-7655774c0241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:46.250741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c22187-3d11-4094-b202-7655774c0241 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:46.263198 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:46.315574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79083b62-3376-434f-b056-f48cd7a90fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:46.316308 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79083b62-3376-434f-b056-f48cd7a90fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:46.316669 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79083b62-3376-434f-b056-f48cd7a90fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:46.317022 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79083b62-3376-434f-b056-f48cd7a90fbf admin admin] PATH_INFO: `/v3/users/c3bf549dcf834ce1922b1ab29f2d9fba` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:46.318318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79083b62-3376-434f-b056-f48cd7a90fbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3bf549dcf834ce1922b1ab29f2d9fba)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:46.320497 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79083b62-3376-434f-b056-f48cd7a90fbf admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:46.565748 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-79083b62-3376-434f-b056-f48cd7a90fbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3bf549dcf834ce1922b1ab29f2d9fba'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:46.676021 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-79083b62-3376-434f-b056-f48cd7a90fbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3bf549dcf834ce1922b1ab29f2d9fba'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:46.676672 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-79083b62-3376-434f-b056-f48cd7a90fbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3bf549dcf834ce1922b1ab29f2d9fba'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:46.694090 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-79083b62-3376-434f-b056-f48cd7a90fbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3bf549dcf834ce1922b1ab29f2d9fba'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:46.707545 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-79083b62-3376-434f-b056-f48cd7a90fbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3bf549dcf834ce1922b1ab29f2d9fba'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:46.718901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 931/1880] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:35:46 2022] DELETE /identity/v3/users/c3bf549dcf834ce1922b1ab29f2d9fba => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:35:47.014986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba1636-416f-4d3f-a06e-b26b7f8bf551 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:47.017617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-34ba1636-416f-4d3f-a06e-b26b7f8bf551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:47.037356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba1636-416f-4d3f-a06e-b26b7f8bf551 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:47.100903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7193d-749d-48d2-b604-804a354b96bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:47.101727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7193d-749d-48d2-b604-804a354b96bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:47.102087 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7193d-749d-48d2-b604-804a354b96bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:47.102498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7193d-749d-48d2-b604-804a354b96bc admin admin] PATH_INFO: `/v3/projects/d4f8d27ef3d849f2ac867eea735c557e` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:47.103815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7193d-749d-48d2-b604-804a354b96bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4f8d27ef3d849f2ac867eea735c557e)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:47.105812 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7193d-749d-48d2-b604-804a354b96bc admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:47.129245 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:47.259075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ddf7193d-749d-48d2-b604-804a354b96bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4f8d27ef3d849f2ac867eea735c557e', 'request_id': 'req-ddf7193d-749d-48d2-b604-804a354b96bc'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:47.265168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ddf7193d-749d-48d2-b604-804a354b96bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4f8d27ef3d849f2ac867eea735c557e', 'request_id': 'req-ddf7193d-749d-48d2-b604-804a354b96bc'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:47.362080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ddf7193d-749d-48d2-b604-804a354b96bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4f8d27ef3d849f2ac867eea735c557e', 'request_id': 'req-ddf7193d-749d-48d2-b604-804a354b96bc'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:47.372835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ddf7193d-749d-48d2-b604-804a354b96bc admin admin] The token cache is being invalidate because project d4f8d27ef3d849f2ac867eea735c557e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:35:47.373918 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ddf7193d-749d-48d2-b604-804a354b96bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:47.373918 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ddf7193d-749d-48d2-b604-804a354b96bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:47.375470 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 950/1881] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:35:47 2022] DELETE /identity/v3/projects/d4f8d27ef3d849f2ac867eea735c557e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:35:47.380961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45f8dd-f10b-4241-a970-ac4895cac4ea None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:47.381245 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45f8dd-f10b-4241-a970-ac4895cac4ea None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:47.381521 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45f8dd-f10b-4241-a970-ac4895cac4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:47.401037 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-9c45f8dd-f10b-4241-a970-ac4895cac4ea None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:35:47.402238 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9c45f8dd-f10b-4241-a970-ac4895cac4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:47.460760 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 932/1882] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:35:47.468765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9442d-be95-41ec-a22f-4067f52fdc07 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:47.469020 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9442d-be95-41ec-a22f-4067f52fdc07 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:47.469298 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9442d-be95-41ec-a22f-4067f52fdc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:47.489673 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-3ec9442d-be95-41ec-a22f-4067f52fdc07 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:35:47.490925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-3ec9442d-be95-41ec-a22f-4067f52fdc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:47.500187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 951/1883] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:35:47.503866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45f8dd-f10b-4241-a970-ac4895cac4ea None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:47.513533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45f8dd-f10b-4241-a970-ac4895cac4ea None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:47.518101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2d1c4-6302-41c0-a194-14eb2a3060e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:47.518706 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2d1c4-6302-41c0-a194-14eb2a3060e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:47.519020 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2d1c4-6302-41c0-a194-14eb2a3060e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:47.519302 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2d1c4-6302-41c0-a194-14eb2a3060e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:47.519955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2d1c4-6302-41c0-a194-14eb2a3060e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:35:47.520277 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2d1c4-6302-41c0-a194-14eb2a3060e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:47.522664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2d1c4-6302-41c0-a194-14eb2a3060e9 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:47.535241 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 933/1884] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:35:47.540022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9442d-be95-41ec-a22f-4067f52fdc07 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:47.549720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9442d-be95-41ec-a22f-4067f52fdc07 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:47.554573 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f471ae-c4a8-433b-9348-9b488fc04b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:47.555315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f471ae-c4a8-433b-9348-9b488fc04b68 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:47.555666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f471ae-c4a8-433b-9348-9b488fc04b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:47.556011 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f471ae-c4a8-433b-9348-9b488fc04b68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:47.556732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f471ae-c4a8-433b-9348-9b488fc04b68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:47.559076 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f471ae-c4a8-433b-9348-9b488fc04b68 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:47.658284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 952/1885] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:35:47 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:35:47.662335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2d1c4-6302-41c0-a194-14eb2a3060e9 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:47.672428 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2d1c4-6302-41c0-a194-14eb2a3060e9 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:47.691149 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:47.728422 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f785fb-6024-48c4-a9e0-63a5b0fd8928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:47.729040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f785fb-6024-48c4-a9e0-63a5b0fd8928 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:47.729349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f785fb-6024-48c4-a9e0-63a5b0fd8928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:47.729655 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f785fb-6024-48c4-a9e0-63a5b0fd8928 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:47.730390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f785fb-6024-48c4-a9e0-63a5b0fd8928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:47.732365 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f785fb-6024-48c4-a9e0-63a5b0fd8928 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:47.828390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 934/1886] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:35:47 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:35:47.832592 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f471ae-c4a8-433b-9348-9b488fc04b68 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:47.843407 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f471ae-c4a8-433b-9348-9b488fc04b68 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:47.848521 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d37040-db48-4dec-8d97-38a9e1c452ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:47.849124 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d37040-db48-4dec-8d97-38a9e1c452ce admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:47.849484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d37040-db48-4dec-8d97-38a9e1c452ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:47.849766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d37040-db48-4dec-8d97-38a9e1c452ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:47.850433 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d37040-db48-4dec-8d97-38a9e1c452ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:47.852087 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d37040-db48-4dec-8d97-38a9e1c452ce admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:47.862655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 953/1887] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:35:47.866536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f785fb-6024-48c4-a9e0-63a5b0fd8928 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:47.876465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f785fb-6024-48c4-a9e0-63a5b0fd8928 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:47.881356 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127bf77-10e8-42c7-9ae8-895360606adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:47.881949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127bf77-10e8-42c7-9ae8-895360606adb admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:47.882228 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127bf77-10e8-42c7-9ae8-895360606adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:47.882497 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127bf77-10e8-42c7-9ae8-895360606adb admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:47.883169 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127bf77-10e8-42c7-9ae8-895360606adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:47.884815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127bf77-10e8-42c7-9ae8-895360606adb admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:47.895798 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 935/1888] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:35:47.899661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d37040-db48-4dec-8d97-38a9e1c452ce admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:47.910170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d37040-db48-4dec-8d97-38a9e1c452ce admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:47.914943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33380779-c14b-4438-adb8-47906fd3c2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:47.917238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33380779-c14b-4438-adb8-47906fd3c2ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:47.917238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33380779-c14b-4438-adb8-47906fd3c2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:47.917238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33380779-c14b-4438-adb8-47906fd3c2ac admin admin] PATH_INFO: `/v3/projects/294688e0ee5c4237908e4ada300cff84/users/8505452fedfd45c7b7ef912860fc5446/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:47.931814 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33380779-c14b-4438-adb8-47906fd3c2ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=294688e0ee5c4237908e4ada300cff84, user_id=8505452fedfd45c7b7ef912860fc5446, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:47.934444 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33380779-c14b-4438-adb8-47906fd3c2ac admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:48.029160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 954/1889] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:35:47 2022] PUT /identity/v3/projects/294688e0ee5c4237908e4ada300cff84/users/8505452fedfd45c7b7ef912860fc5446/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:35:48.041168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32616db4-0f7f-4109-bef4-4540fa9c8333 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:48.041469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32616db4-0f7f-4109-bef4-4540fa9c8333 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:48.044243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32616db4-0f7f-4109-bef4-4540fa9c8333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:48.078045 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-32616db4-0f7f-4109-bef4-4540fa9c8333 None None] MFA Rules not processed for user `8505452fedfd45c7b7ef912860fc5446`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:35:48.078045 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-32616db4-0f7f-4109-bef4-4540fa9c8333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:48.153468 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 936/1890] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:35:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:35:48.165783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33380779-c14b-4438-adb8-47906fd3c2ac admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:48.167491 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-33380779-c14b-4438-adb8-47906fd3c2ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:48.225958 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33380779-c14b-4438-adb8-47906fd3c2ac admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:48.229695 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607070c7-0764-4b44-95ad-5952ce74adba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:48.230306 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607070c7-0764-4b44-95ad-5952ce74adba service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:48.230590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607070c7-0764-4b44-95ad-5952ce74adba service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:48.230895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607070c7-0764-4b44-95ad-5952ce74adba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:48.239158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607070c7-0764-4b44-95ad-5952ce74adba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:48.240625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607070c7-0764-4b44-95ad-5952ce74adba service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:48.249512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 955/1891] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:35:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:35:48.271628 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32616db4-0f7f-4109-bef4-4540fa9c8333 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:48.335243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32616db4-0f7f-4109-bef4-4540fa9c8333 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:48.340322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639ee4f-484f-4f8c-b0a1-ebcf9c87ca8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:48.340962 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639ee4f-484f-4f8c-b0a1-ebcf9c87ca8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:48.341272 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639ee4f-484f-4f8c-b0a1-ebcf9c87ca8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:48.341603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639ee4f-484f-4f8c-b0a1-ebcf9c87ca8c admin admin] PATH_INFO: `/v3/users/8505452fedfd45c7b7ef912860fc5446` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:48.342749 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639ee4f-484f-4f8c-b0a1-ebcf9c87ca8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8505452fedfd45c7b7ef912860fc5446)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:48.344754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639ee4f-484f-4f8c-b0a1-ebcf9c87ca8c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:48.568592 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:48.581269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e639ee4f-484f-4f8c-b0a1-ebcf9c87ca8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8505452fedfd45c7b7ef912860fc5446'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:48.696816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e639ee4f-484f-4f8c-b0a1-ebcf9c87ca8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8505452fedfd45c7b7ef912860fc5446'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:48.697869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e639ee4f-484f-4f8c-b0a1-ebcf9c87ca8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8505452fedfd45c7b7ef912860fc5446'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:48.716191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e639ee4f-484f-4f8c-b0a1-ebcf9c87ca8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8505452fedfd45c7b7ef912860fc5446'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:48.723433 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e639ee4f-484f-4f8c-b0a1-ebcf9c87ca8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8505452fedfd45c7b7ef912860fc5446'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:48.730167 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 937/1892] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:35:48 2022] DELETE /identity/v3/users/8505452fedfd45c7b7ef912860fc5446 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:35:48.735764 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e12eda-641e-4121-96b5-3ced254f719c None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:48.736200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e12eda-641e-4121-96b5-3ced254f719c None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:48.736683 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e12eda-641e-4121-96b5-3ced254f719c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:48.763901 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-20e12eda-641e-4121-96b5-3ced254f719c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:35:48.765259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-20e12eda-641e-4121-96b5-3ced254f719c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:48.811717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:48.831023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 956/1893] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:35:48.845586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639ee4f-484f-4f8c-b0a1-ebcf9c87ca8c admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:48.847147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e639ee4f-484f-4f8c-b0a1-ebcf9c87ca8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:48.905205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639ee4f-484f-4f8c-b0a1-ebcf9c87ca8c admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:48.908971 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fa22b-d8d5-44ef-9beb-f2a741df8521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:48.909582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fa22b-d8d5-44ef-9beb-f2a741df8521 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:48.909870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fa22b-d8d5-44ef-9beb-f2a741df8521 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:48.910143 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fa22b-d8d5-44ef-9beb-f2a741df8521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:48.919198 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fa22b-d8d5-44ef-9beb-f2a741df8521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:48.920677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fa22b-d8d5-44ef-9beb-f2a741df8521 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:48.930740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 938/1894] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:35:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:35:49.463785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e12eda-641e-4121-96b5-3ced254f719c None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:49.466016 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-20e12eda-641e-4121-96b5-3ced254f719c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:49.478494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e12eda-641e-4121-96b5-3ced254f719c None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:49.483169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8aa3fe-d538-4f84-9ad6-2555b330d018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:49.483804 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8aa3fe-d538-4f84-9ad6-2555b330d018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:49.484074 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8aa3fe-d538-4f84-9ad6-2555b330d018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:49.484346 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8aa3fe-d538-4f84-9ad6-2555b330d018 admin admin] PATH_INFO: `/v3/projects/294688e0ee5c4237908e4ada300cff84` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:49.486058 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8aa3fe-d538-4f84-9ad6-2555b330d018 admin admin] RBAC: Authorizing `identity:delete_project(project_id=294688e0ee5c4237908e4ada300cff84)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:49.487982 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8aa3fe-d538-4f84-9ad6-2555b330d018 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:49.664994 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-4f8aa3fe-d538-4f84-9ad6-2555b330d018 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '294688e0ee5c4237908e4ada300cff84', 'request_id': 'req-4f8aa3fe-d538-4f84-9ad6-2555b330d018'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:49.674072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-4f8aa3fe-d538-4f84-9ad6-2555b330d018 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '294688e0ee5c4237908e4ada300cff84', 'request_id': 'req-4f8aa3fe-d538-4f84-9ad6-2555b330d018'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:49.785403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-4f8aa3fe-d538-4f84-9ad6-2555b330d018 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '294688e0ee5c4237908e4ada300cff84', 'request_id': 'req-4f8aa3fe-d538-4f84-9ad6-2555b330d018'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:49.800208 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-4f8aa3fe-d538-4f84-9ad6-2555b330d018 admin admin] The token cache is being invalidate because project 294688e0ee5c4237908e4ada300cff84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:35:49.800765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-4f8aa3fe-d538-4f84-9ad6-2555b330d018 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:49.801479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-4f8aa3fe-d538-4f84-9ad6-2555b330d018 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:35:49.803034 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 957/1895] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:35:49 2022] DELETE /identity/v3/projects/294688e0ee5c4237908e4ada300cff84 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:35:49.807976 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5b841-677f-42ec-bdbf-bbb33cf5c996 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:49.808489 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5b841-677f-42ec-bdbf-bbb33cf5c996 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:49.809162 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5b841-677f-42ec-bdbf-bbb33cf5c996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:49.837608 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-45c5b841-677f-42ec-bdbf-bbb33cf5c996 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:35:49.839141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-45c5b841-677f-42ec-bdbf-bbb33cf5c996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:49.908383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 939/1896] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:35:49.916481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caf8cc-92b4-4e98-809d-4e7b63a917ea None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:49.916798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caf8cc-92b4-4e98-809d-4e7b63a917ea None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:49.917059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caf8cc-92b4-4e98-809d-4e7b63a917ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:49.980269 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-c8caf8cc-92b4-4e98-809d-4e7b63a917ea None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:35:49.981547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c8caf8cc-92b4-4e98-809d-4e7b63a917ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:49.995619 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 958/1897] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:35:50.000023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5b841-677f-42ec-bdbf-bbb33cf5c996 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:50.011112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5b841-677f-42ec-bdbf-bbb33cf5c996 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:50.016664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667446d-79b7-456c-9432-65768f4763b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:50.017295 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667446d-79b7-456c-9432-65768f4763b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:50.017574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667446d-79b7-456c-9432-65768f4763b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:50.017849 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667446d-79b7-456c-9432-65768f4763b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:50.018537 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667446d-79b7-456c-9432-65768f4763b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:35:50.018880 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667446d-79b7-456c-9432-65768f4763b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:50.020539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667446d-79b7-456c-9432-65768f4763b4 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:50.035738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 940/1898] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:35:50.039793 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8caf8cc-92b4-4e98-809d-4e7b63a917ea None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:50.051571 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8caf8cc-92b4-4e98-809d-4e7b63a917ea None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:50.056688 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232dcfd-23fa-4435-b2c2-621dd71a81c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:50.059534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232dcfd-23fa-4435-b2c2-621dd71a81c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:50.059534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232dcfd-23fa-4435-b2c2-621dd71a81c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:50.059534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232dcfd-23fa-4435-b2c2-621dd71a81c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:50.059534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232dcfd-23fa-4435-b2c2-621dd71a81c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:50.060736 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232dcfd-23fa-4435-b2c2-621dd71a81c6 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:50.217088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 959/1899] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:35:50 2022] POST /identity/v3/projects => generated 375 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:35:50.221796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667446d-79b7-456c-9432-65768f4763b4 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:50.242874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667446d-79b7-456c-9432-65768f4763b4 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:50.307459 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:35:50.329966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80038a18-634b-430d-bbe7-748a9a848cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:50.330590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80038a18-634b-430d-bbe7-748a9a848cfa admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:50.330900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80038a18-634b-430d-bbe7-748a9a848cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:50.331179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80038a18-634b-430d-bbe7-748a9a848cfa admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:50.331815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80038a18-634b-430d-bbe7-748a9a848cfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:50.333739 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80038a18-634b-430d-bbe7-748a9a848cfa admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:50.436664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 941/1900] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:35:50 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:35:50.441402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232dcfd-23fa-4435-b2c2-621dd71a81c6 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:50.451760 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232dcfd-23fa-4435-b2c2-621dd71a81c6 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:50.456464 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56cae0-85cd-418c-bd75-0d0918e33a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:50.457076 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56cae0-85cd-418c-bd75-0d0918e33a1a admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:50.462252 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56cae0-85cd-418c-bd75-0d0918e33a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:50.467989 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56cae0-85cd-418c-bd75-0d0918e33a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:50.467989 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56cae0-85cd-418c-bd75-0d0918e33a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:50.484502 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56cae0-85cd-418c-bd75-0d0918e33a1a admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:50.511562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 960/1901] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:35:50.516101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80038a18-634b-430d-bbe7-748a9a848cfa admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:50.527447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80038a18-634b-430d-bbe7-748a9a848cfa admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:50.538836 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab68dc9-9dea-402f-bd21-b3141b83db93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:50.538836 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab68dc9-9dea-402f-bd21-b3141b83db93 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:50.538836 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab68dc9-9dea-402f-bd21-b3141b83db93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:50.538836 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab68dc9-9dea-402f-bd21-b3141b83db93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:50.539280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab68dc9-9dea-402f-bd21-b3141b83db93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:50.539280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab68dc9-9dea-402f-bd21-b3141b83db93 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:50.548190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 942/1902] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:35:50.552414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56cae0-85cd-418c-bd75-0d0918e33a1a admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:50.563250 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56cae0-85cd-418c-bd75-0d0918e33a1a admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:50.568735 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32964bc-8365-40a4-8648-911df780adc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:50.570135 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32964bc-8365-40a4-8648-911df780adc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:50.570426 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32964bc-8365-40a4-8648-911df780adc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:50.570701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32964bc-8365-40a4-8648-911df780adc5 admin admin] PATH_INFO: `/v3/projects/67e9c990eaf841398a915b6bab60c292/users/dea3dd94a500448a9f5c480e5ea74535/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:50.606381 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32964bc-8365-40a4-8648-911df780adc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67e9c990eaf841398a915b6bab60c292, user_id=dea3dd94a500448a9f5c480e5ea74535, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:50.609682 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32964bc-8365-40a4-8648-911df780adc5 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:50.743269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86464383-df79-4385-9d68-0e43bf6466d9 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:50.743671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86464383-df79-4385-9d68-0e43bf6466d9 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:50.744028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86464383-df79-4385-9d68-0e43bf6466d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:50.746914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 961/1903] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:35:50 2022] PUT /identity/v3/projects/67e9c990eaf841398a915b6bab60c292/users/dea3dd94a500448a9f5c480e5ea74535/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:35:50.799926 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-86464383-df79-4385-9d68-0e43bf6466d9 None None] MFA Rules not processed for user `dea3dd94a500448a9f5c480e5ea74535`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:35:50.799926 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-86464383-df79-4385-9d68-0e43bf6466d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:50.890942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 943/1904] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:35:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:35:50.903673 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32964bc-8365-40a4-8648-911df780adc5 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:35:50.904770 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b32964bc-8365-40a4-8648-911df780adc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:35:50.967860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32964bc-8365-40a4-8648-911df780adc5 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:35:50.971599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbef6f-22da-48f8-8f0b-dd95ca787824 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:35:50.972368 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbef6f-22da-48f8-8f0b-dd95ca787824 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:35:50.972743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbef6f-22da-48f8-8f0b-dd95ca787824 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:35:50.973107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbef6f-22da-48f8-8f0b-dd95ca787824 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:35:50.983164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbef6f-22da-48f8-8f0b-dd95ca787824 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:35:50.983164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbef6f-22da-48f8-8f0b-dd95ca787824 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:35:50.992558 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 962/1905] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:35:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:00.988877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:01.705542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=48,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:13.769362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86464383-df79-4385-9d68-0e43bf6466d9 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:13.830377 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86464383-df79-4385-9d68-0e43bf6466d9 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:13.835249 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0554aa9-e66b-4047-82f1-c445818afc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:13.835976 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0554aa9-e66b-4047-82f1-c445818afc3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:13.836256 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0554aa9-e66b-4047-82f1-c445818afc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:13.836536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0554aa9-e66b-4047-82f1-c445818afc3c admin admin] PATH_INFO: `/v3/users/dea3dd94a500448a9f5c480e5ea74535` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:13.837653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0554aa9-e66b-4047-82f1-c445818afc3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dea3dd94a500448a9f5c480e5ea74535)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:13.839759 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0554aa9-e66b-4047-82f1-c445818afc3c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:14.033155 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a0554aa9-e66b-4047-82f1-c445818afc3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dea3dd94a500448a9f5c480e5ea74535'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:14.183044 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a0554aa9-e66b-4047-82f1-c445818afc3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dea3dd94a500448a9f5c480e5ea74535'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:14.183044 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a0554aa9-e66b-4047-82f1-c445818afc3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dea3dd94a500448a9f5c480e5ea74535'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:14.192522 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a0554aa9-e66b-4047-82f1-c445818afc3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dea3dd94a500448a9f5c480e5ea74535'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:14.199947 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a0554aa9-e66b-4047-82f1-c445818afc3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dea3dd94a500448a9f5c480e5ea74535'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:14.206693 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 944/1906] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:13 2022] DELETE /identity/v3/users/dea3dd94a500448a9f5c480e5ea74535 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:14.211362 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ef599-46d4-4cc9-824d-38fb4d0c9163 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:14.211846 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ef599-46d4-4cc9-824d-38fb4d0c9163 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:14.212027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ef599-46d4-4cc9-824d-38fb4d0c9163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:14.231829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-fd9ef599-46d4-4cc9-824d-38fb4d0c9163 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:14.233105 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-fd9ef599-46d4-4cc9-824d-38fb4d0c9163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:14.295528 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 963/1907] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:14.305382 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0554aa9-e66b-4047-82f1-c445818afc3c admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:14.306845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a0554aa9-e66b-4047-82f1-c445818afc3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:14.367595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0554aa9-e66b-4047-82f1-c445818afc3c admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:14.371253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cae68d-2531-4335-92cd-745d59806e80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:14.372030 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cae68d-2531-4335-92cd-745d59806e80 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:14.372334 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cae68d-2531-4335-92cd-745d59806e80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:14.372596 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cae68d-2531-4335-92cd-745d59806e80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:14.381575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cae68d-2531-4335-92cd-745d59806e80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:14.383053 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cae68d-2531-4335-92cd-745d59806e80 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:14.393572 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 945/1908] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:14.866594 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ef599-46d4-4cc9-824d-38fb4d0c9163 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:14.868233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-fd9ef599-46d4-4cc9-824d-38fb4d0c9163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:14.882552 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ef599-46d4-4cc9-824d-38fb4d0c9163 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:14.887476 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7dba0-d677-4111-8512-05091206b6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:14.897745 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7dba0-d677-4111-8512-05091206b6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:14.898054 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7dba0-d677-4111-8512-05091206b6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:14.898344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7dba0-d677-4111-8512-05091206b6e0 admin admin] PATH_INFO: `/v3/projects/67e9c990eaf841398a915b6bab60c292` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:14.899670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7dba0-d677-4111-8512-05091206b6e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67e9c990eaf841398a915b6bab60c292)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:14.902242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7dba0-d677-4111-8512-05091206b6e0 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:15.131602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-86b7dba0-d677-4111-8512-05091206b6e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67e9c990eaf841398a915b6bab60c292', 'request_id': 'req-86b7dba0-d677-4111-8512-05091206b6e0'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:15.138361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-86b7dba0-d677-4111-8512-05091206b6e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67e9c990eaf841398a915b6bab60c292', 'request_id': 'req-86b7dba0-d677-4111-8512-05091206b6e0'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:15.227605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-86b7dba0-d677-4111-8512-05091206b6e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67e9c990eaf841398a915b6bab60c292', 'request_id': 'req-86b7dba0-d677-4111-8512-05091206b6e0'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:15.239425 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-86b7dba0-d677-4111-8512-05091206b6e0 admin admin] The token cache is being invalidate because project 67e9c990eaf841398a915b6bab60c292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:15.240281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-86b7dba0-d677-4111-8512-05091206b6e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:15.240476 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-86b7dba0-d677-4111-8512-05091206b6e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:15.241972 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 964/1909] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:14 2022] DELETE /identity/v3/projects/67e9c990eaf841398a915b6bab60c292 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:15.246712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebab594-0d18-4a79-9d79-b32565d36e5a None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:15.247257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebab594-0d18-4a79-9d79-b32565d36e5a None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:15.247366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebab594-0d18-4a79-9d79-b32565d36e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:15.267056 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-5ebab594-0d18-4a79-9d79-b32565d36e5a None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:15.269013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-5ebab594-0d18-4a79-9d79-b32565d36e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:15.285834 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:15.326600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 946/1910] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:15.335480 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ef256-ec0e-4c66-91ca-6f789dca8a10 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:15.344985 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ef256-ec0e-4c66-91ca-6f789dca8a10 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:15.344985 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ef256-ec0e-4c66-91ca-6f789dca8a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:15.357238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-b06ef256-ec0e-4c66-91ca-6f789dca8a10 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:15.358515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b06ef256-ec0e-4c66-91ca-6f789dca8a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:15.367603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 965/1911] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:15.371955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebab594-0d18-4a79-9d79-b32565d36e5a None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:15.382650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebab594-0d18-4a79-9d79-b32565d36e5a None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:15.387505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c41192-766c-4e2a-89bc-30ec1d76631c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:15.388197 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c41192-766c-4e2a-89bc-30ec1d76631c admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:15.388786 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c41192-766c-4e2a-89bc-30ec1d76631c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:15.389316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c41192-766c-4e2a-89bc-30ec1d76631c admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:15.390025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c41192-766c-4e2a-89bc-30ec1d76631c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:36:15.390809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c41192-766c-4e2a-89bc-30ec1d76631c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:15.392509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c41192-766c-4e2a-89bc-30ec1d76631c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:15.404235 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 947/1912] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:36:15.409111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ef256-ec0e-4c66-91ca-6f789dca8a10 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:15.418482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ef256-ec0e-4c66-91ca-6f789dca8a10 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:15.423283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858189ec-5228-4e96-9da2-bbf624847824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:15.424089 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858189ec-5228-4e96-9da2-bbf624847824 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:15.424375 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858189ec-5228-4e96-9da2-bbf624847824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:15.424655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858189ec-5228-4e96-9da2-bbf624847824 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:15.425290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858189ec-5228-4e96-9da2-bbf624847824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:15.427297 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858189ec-5228-4e96-9da2-bbf624847824 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:15.532665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 966/1913] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:15 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:15.536750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c41192-766c-4e2a-89bc-30ec1d76631c admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:15.546701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c41192-766c-4e2a-89bc-30ec1d76631c admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:15.582242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858189ec-5228-4e96-9da2-bbf624847824 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:15.584035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-858189ec-5228-4e96-9da2-bbf624847824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:15.616517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b37b5-7287-4b25-90a9-a66486f4e3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:15.620186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b37b5-7287-4b25-90a9-a66486f4e3d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:15.620186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b37b5-7287-4b25-90a9-a66486f4e3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:15.620186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b37b5-7287-4b25-90a9-a66486f4e3d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:15.620186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b37b5-7287-4b25-90a9-a66486f4e3d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:15.621541 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b37b5-7287-4b25-90a9-a66486f4e3d7 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:15.656350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:15.662119 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858189ec-5228-4e96-9da2-bbf624847824 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:15.667827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ce995-cb59-4fdd-8cf0-cc81178b1ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:15.668490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ce995-cb59-4fdd-8cf0-cc81178b1ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:15.668887 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ce995-cb59-4fdd-8cf0-cc81178b1ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:15.669175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ce995-cb59-4fdd-8cf0-cc81178b1ede admin admin] PATH_INFO: `/v3/users/6fdd0b30046e448da0f16f4aafd8d1bd` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:15.670324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ce995-cb59-4fdd-8cf0-cc81178b1ede admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fdd0b30046e448da0f16f4aafd8d1bd)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:15.672813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ce995-cb59-4fdd-8cf0-cc81178b1ede admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:15.727487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 948/1914] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:15 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:15.731488 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b37b5-7287-4b25-90a9-a66486f4e3d7 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:15.741672 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b37b5-7287-4b25-90a9-a66486f4e3d7 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:15.747185 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a16c92-18ea-44ad-90a1-b7c879c08735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:15.747562 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a16c92-18ea-44ad-90a1-b7c879c08735 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:15.747846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a16c92-18ea-44ad-90a1-b7c879c08735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:15.748235 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a16c92-18ea-44ad-90a1-b7c879c08735 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:15.748949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a16c92-18ea-44ad-90a1-b7c879c08735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:15.751118 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a16c92-18ea-44ad-90a1-b7c879c08735 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:15.761953 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 949/1915] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:15.765907 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a16c92-18ea-44ad-90a1-b7c879c08735 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:15.776381 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a16c92-18ea-44ad-90a1-b7c879c08735 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:15.781188 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb34ee-c33a-4f8f-939e-1d23246fdd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:15.781846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb34ee-c33a-4f8f-939e-1d23246fdd76 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:15.782199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb34ee-c33a-4f8f-939e-1d23246fdd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:15.782545 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb34ee-c33a-4f8f-939e-1d23246fdd76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:15.783331 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb34ee-c33a-4f8f-939e-1d23246fdd76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:15.785034 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb34ee-c33a-4f8f-939e-1d23246fdd76 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:15.798839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 950/1916] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:15.800225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb34ee-c33a-4f8f-939e-1d23246fdd76 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:15.810032 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb34ee-c33a-4f8f-939e-1d23246fdd76 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:15.815554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c12eb6-2f14-4c30-b1d1-f683d5a51990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:15.816247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c12eb6-2f14-4c30-b1d1-f683d5a51990 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:15.816535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c12eb6-2f14-4c30-b1d1-f683d5a51990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:15.816816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c12eb6-2f14-4c30-b1d1-f683d5a51990 admin admin] PATH_INFO: `/v3/projects/fc60ee38517541b8bc153807f5332011/users/e8429576e286422cb09591897e82cd69/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:15.833515 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c12eb6-2f14-4c30-b1d1-f683d5a51990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc60ee38517541b8bc153807f5332011, user_id=e8429576e286422cb09591897e82cd69, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:15.836423 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c12eb6-2f14-4c30-b1d1-f683d5a51990 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:15.950601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 951/1917] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:15 2022] PUT /identity/v3/projects/fc60ee38517541b8bc153807f5332011/users/e8429576e286422cb09591897e82cd69/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:15.955916 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a6160-47a2-40c7-9c03-0ade6337c912 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:15.956391 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a6160-47a2-40c7-9c03-0ade6337c912 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:15.956906 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a6160-47a2-40c7-9c03-0ade6337c912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:15.961622 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-181ce995-cb59-4fdd-8cf0-cc81178b1ede admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fdd0b30046e448da0f16f4aafd8d1bd'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:15.978433 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-ce4a6160-47a2-40c7-9c03-0ade6337c912 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:15.989442 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ce4a6160-47a2-40c7-9c03-0ade6337c912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:16.028674 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:16.039412 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-181ce995-cb59-4fdd-8cf0-cc81178b1ede admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fdd0b30046e448da0f16f4aafd8d1bd'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:16.041482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 952/1918] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:16.042342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a6160-47a2-40c7-9c03-0ade6337c912 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:16.052886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ce4a6160-47a2-40c7-9c03-0ade6337c912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:16.057570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-181ce995-cb59-4fdd-8cf0-cc81178b1ede admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fdd0b30046e448da0f16f4aafd8d1bd'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:16.058348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-181ce995-cb59-4fdd-8cf0-cc81178b1ede admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fdd0b30046e448da0f16f4aafd8d1bd'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:16.060105 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a6160-47a2-40c7-9c03-0ade6337c912 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:16.068162 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-181ce995-cb59-4fdd-8cf0-cc81178b1ede admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fdd0b30046e448da0f16f4aafd8d1bd'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:16.068318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9c47c-434e-476b-b1a4-913815d13267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:16.068318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9c47c-434e-476b-b1a4-913815d13267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:16.068318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9c47c-434e-476b-b1a4-913815d13267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:16.068318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9c47c-434e-476b-b1a4-913815d13267 admin admin] PATH_INFO: `/v3/users/3ff06d0b603242f990e59ae898a409dd` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:16.069089 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9c47c-434e-476b-b1a4-913815d13267 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ff06d0b603242f990e59ae898a409dd)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:16.071054 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9c47c-434e-476b-b1a4-913815d13267 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:16.077868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 967/1919] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:15 2022] DELETE /identity/v3/users/6fdd0b30046e448da0f16f4aafd8d1bd => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:16.077868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ce995-cb59-4fdd-8cf0-cc81178b1ede admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:16.077868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-181ce995-cb59-4fdd-8cf0-cc81178b1ede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:16.143106 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ce995-cb59-4fdd-8cf0-cc81178b1ede admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:16.146812 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0029b-62aa-45a1-8e50-e729b7b42379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:16.147458 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0029b-62aa-45a1-8e50-e729b7b42379 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:16.147743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0029b-62aa-45a1-8e50-e729b7b42379 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:16.148058 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0029b-62aa-45a1-8e50-e729b7b42379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:16.149794 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-3ff0029b-62aa-45a1-8e50-e729b7b42379 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:16.214146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0029b-62aa-45a1-8e50-e729b7b42379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:16.215874 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0029b-62aa-45a1-8e50-e729b7b42379 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:16.225329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 968/1920] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:16.227024 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae12ba1-4f94-48b4-9518-2f8a8c479f96 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:16.227024 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae12ba1-4f94-48b4-9518-2f8a8c479f96 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:16.227314 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae12ba1-4f94-48b4-9518-2f8a8c479f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:16.246051 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-9ae12ba1-4f94-48b4-9518-2f8a8c479f96 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:16.247382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-9ae12ba1-4f94-48b4-9518-2f8a8c479f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:16.256150 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 969/1921] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:16.265930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae12ba1-4f94-48b4-9518-2f8a8c479f96 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:16.268062 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-9ae12ba1-4f94-48b4-9518-2f8a8c479f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:16.275532 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a9d9c47c-434e-476b-b1a4-913815d13267 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ff06d0b603242f990e59ae898a409dd'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:16.280616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae12ba1-4f94-48b4-9518-2f8a8c479f96 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:16.284316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abeab54-26ff-4e4e-bf1f-8b0d402efbe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:16.285056 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeab54-26ff-4e4e-bf1f-8b0d402efbe8 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:16.285269 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeab54-26ff-4e4e-bf1f-8b0d402efbe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:16.285541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeab54-26ff-4e4e-bf1f-8b0d402efbe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:16.295209 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abeab54-26ff-4e4e-bf1f-8b0d402efbe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:16.297502 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abeab54-26ff-4e4e-bf1f-8b0d402efbe8 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:16.306813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 970/1922] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:36:16.405525 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a9d9c47c-434e-476b-b1a4-913815d13267 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ff06d0b603242f990e59ae898a409dd'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:16.408922 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a9d9c47c-434e-476b-b1a4-913815d13267 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ff06d0b603242f990e59ae898a409dd'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:16.424128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a9d9c47c-434e-476b-b1a4-913815d13267 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ff06d0b603242f990e59ae898a409dd'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:16.431398 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a9d9c47c-434e-476b-b1a4-913815d13267 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ff06d0b603242f990e59ae898a409dd'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:16.437614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 953/1923] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:16 2022] DELETE /identity/v3/users/3ff06d0b603242f990e59ae898a409dd => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:16.442655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26336462-5127-4828-a07a-830d744df135 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:16.442984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26336462-5127-4828-a07a-830d744df135 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:16.448802 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26336462-5127-4828-a07a-830d744df135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:16.451810 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:16.462729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-26336462-5127-4828-a07a-830d744df135 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:16.464040 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-26336462-5127-4828-a07a-830d744df135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:16.529049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 971/1924] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:16.538760 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9c47c-434e-476b-b1a4-913815d13267 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:16.540305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a9d9c47c-434e-476b-b1a4-913815d13267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:16.613076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9c47c-434e-476b-b1a4-913815d13267 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:16.617001 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbb388-c8fe-4722-b01d-5eb1ddf6cde9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:16.617563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbb388-c8fe-4722-b01d-5eb1ddf6cde9 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:16.617852 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbb388-c8fe-4722-b01d-5eb1ddf6cde9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:16.618132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbb388-c8fe-4722-b01d-5eb1ddf6cde9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:16.628387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacbb388-c8fe-4722-b01d-5eb1ddf6cde9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:16.629854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacbb388-c8fe-4722-b01d-5eb1ddf6cde9 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:16.640081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 954/1925] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:16.819706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26336462-5127-4828-a07a-830d744df135 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:16.821187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-26336462-5127-4828-a07a-830d744df135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:16.834522 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26336462-5127-4828-a07a-830d744df135 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:16.839847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77520b3-66c3-45a2-88af-1d7142c954b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:16.840746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77520b3-66c3-45a2-88af-1d7142c954b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:16.841033 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77520b3-66c3-45a2-88af-1d7142c954b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:16.841308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77520b3-66c3-45a2-88af-1d7142c954b1 admin admin] PATH_INFO: `/v3/projects/3c30822e89fe4cc7a444dee7c417c511` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:16.842445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77520b3-66c3-45a2-88af-1d7142c954b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c30822e89fe4cc7a444dee7c417c511)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:16.844996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77520b3-66c3-45a2-88af-1d7142c954b1 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:17.083700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d77520b3-66c3-45a2-88af-1d7142c954b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c30822e89fe4cc7a444dee7c417c511', 'request_id': 'req-d77520b3-66c3-45a2-88af-1d7142c954b1'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:17.090171 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d77520b3-66c3-45a2-88af-1d7142c954b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c30822e89fe4cc7a444dee7c417c511', 'request_id': 'req-d77520b3-66c3-45a2-88af-1d7142c954b1'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:17.285885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d77520b3-66c3-45a2-88af-1d7142c954b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c30822e89fe4cc7a444dee7c417c511', 'request_id': 'req-d77520b3-66c3-45a2-88af-1d7142c954b1'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:17.300039 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d77520b3-66c3-45a2-88af-1d7142c954b1 admin admin] The token cache is being invalidate because project 3c30822e89fe4cc7a444dee7c417c511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:17.300329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d77520b3-66c3-45a2-88af-1d7142c954b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:17.301002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d77520b3-66c3-45a2-88af-1d7142c954b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:17.302551 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 972/1926] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:16 2022] DELETE /identity/v3/projects/3c30822e89fe4cc7a444dee7c417c511 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:17.306513 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbb388-c8fe-4722-b01d-5eb1ddf6cde9 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:17.318819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbb388-c8fe-4722-b01d-5eb1ddf6cde9 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:17.380629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f7986-23a7-42f8-a394-499fb21d6226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:17.381268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f7986-23a7-42f8-a394-499fb21d6226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:17.381555 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f7986-23a7-42f8-a394-499fb21d6226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:17.382013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f7986-23a7-42f8-a394-499fb21d6226 admin admin] PATH_INFO: `/v3/users/d0713340ad8045afb5c9b2c8427aeeda` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:17.383362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f7986-23a7-42f8-a394-499fb21d6226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0713340ad8045afb5c9b2c8427aeeda)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:17.385530 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f7986-23a7-42f8-a394-499fb21d6226 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:17.394842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77520b3-66c3-45a2-88af-1d7142c954b1 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:17.397309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d77520b3-66c3-45a2-88af-1d7142c954b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:17.400024 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:17.417352 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77520b3-66c3-45a2-88af-1d7142c954b1 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:17.429766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e31d6c-05b0-4f1d-bec6-1cba3f29ec1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:17.431036 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e31d6c-05b0-4f1d-bec6-1cba3f29ec1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:17.431665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e31d6c-05b0-4f1d-bec6-1cba3f29ec1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:17.432236 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e31d6c-05b0-4f1d-bec6-1cba3f29ec1d admin admin] PATH_INFO: `/v3/projects/1d655e61a6b045b38a636aa9b06d3365` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:17.433652 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e31d6c-05b0-4f1d-bec6-1cba3f29ec1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d655e61a6b045b38a636aa9b06d3365)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:17.435652 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e31d6c-05b0-4f1d-bec6-1cba3f29ec1d admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:17.709595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d6e31d6c-05b0-4f1d-bec6-1cba3f29ec1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d655e61a6b045b38a636aa9b06d3365', 'request_id': 'req-d6e31d6c-05b0-4f1d-bec6-1cba3f29ec1d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:17.712140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1e0f7986-23a7-42f8-a394-499fb21d6226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0713340ad8045afb5c9b2c8427aeeda'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:17.717650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d6e31d6c-05b0-4f1d-bec6-1cba3f29ec1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d655e61a6b045b38a636aa9b06d3365', 'request_id': 'req-d6e31d6c-05b0-4f1d-bec6-1cba3f29ec1d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:17.796915 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1e0f7986-23a7-42f8-a394-499fb21d6226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0713340ad8045afb5c9b2c8427aeeda'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:17.798098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1e0f7986-23a7-42f8-a394-499fb21d6226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0713340ad8045afb5c9b2c8427aeeda'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:17.820885 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1e0f7986-23a7-42f8-a394-499fb21d6226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0713340ad8045afb5c9b2c8427aeeda'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:17.832080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1e0f7986-23a7-42f8-a394-499fb21d6226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0713340ad8045afb5c9b2c8427aeeda'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:17.841806 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 955/1927] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:17 2022] DELETE /identity/v3/users/d0713340ad8045afb5c9b2c8427aeeda => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:17.909779 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d6e31d6c-05b0-4f1d-bec6-1cba3f29ec1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d655e61a6b045b38a636aa9b06d3365', 'request_id': 'req-d6e31d6c-05b0-4f1d-bec6-1cba3f29ec1d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:17.924694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d6e31d6c-05b0-4f1d-bec6-1cba3f29ec1d admin admin] The token cache is being invalidate because project 1d655e61a6b045b38a636aa9b06d3365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:17.925003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d6e31d6c-05b0-4f1d-bec6-1cba3f29ec1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:17.925711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d6e31d6c-05b0-4f1d-bec6-1cba3f29ec1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:17.927891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 973/1928] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:17 2022] DELETE /identity/v3/projects/1d655e61a6b045b38a636aa9b06d3365 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:17.932757 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc712e6-64a8-4ad5-a6c9-c8a82af9c17c None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:17.934406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc712e6-64a8-4ad5-a6c9-c8a82af9c17c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:17.934695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc712e6-64a8-4ad5-a6c9-c8a82af9c17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:17.958638 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-9bc712e6-64a8-4ad5-a6c9-c8a82af9c17c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:17.959875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9bc712e6-64a8-4ad5-a6c9-c8a82af9c17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:18.019507 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 956/1929] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:18.027583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759739c2-1ae8-4783-b71d-76a2b21cd854 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:18.028129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759739c2-1ae8-4783-b71d-76a2b21cd854 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:18.035327 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759739c2-1ae8-4783-b71d-76a2b21cd854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:18.049942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-759739c2-1ae8-4783-b71d-76a2b21cd854 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:18.051317 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-759739c2-1ae8-4783-b71d-76a2b21cd854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:18.068136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 974/1930] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:18.072043 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc712e6-64a8-4ad5-a6c9-c8a82af9c17c None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:18.084013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc712e6-64a8-4ad5-a6c9-c8a82af9c17c None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:18.090834 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f43c-496f-4090-a1ef-9a77f549c9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:18.091458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f43c-496f-4090-a1ef-9a77f549c9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:18.091734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f43c-496f-4090-a1ef-9a77f549c9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:18.092011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f43c-496f-4090-a1ef-9a77f549c9c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:18.092681 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609f43c-496f-4090-a1ef-9a77f549c9c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:36:18.093288 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609f43c-496f-4090-a1ef-9a77f549c9c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:18.095397 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609f43c-496f-4090-a1ef-9a77f549c9c5 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:18.114836 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 957/1931] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:36:18.115970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759739c2-1ae8-4783-b71d-76a2b21cd854 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:18.122572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:18.127787 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759739c2-1ae8-4783-b71d-76a2b21cd854 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:18.132887 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d4af3-b236-4dda-80e9-b3d23f1946b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:18.133606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d4af3-b236-4dda-80e9-b3d23f1946b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:18.133954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d4af3-b236-4dda-80e9-b3d23f1946b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:18.134323 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d4af3-b236-4dda-80e9-b3d23f1946b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:18.135070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d4af3-b236-4dda-80e9-b3d23f1946b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:18.137181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d4af3-b236-4dda-80e9-b3d23f1946b3 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:18.273181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 975/1932] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:18 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:18.281247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f43c-496f-4090-a1ef-9a77f549c9c5 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:18.291849 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f43c-496f-4090-a1ef-9a77f549c9c5 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:18.342891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10873914-fc81-4703-bb42-babec053e499 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:18.344761 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10873914-fc81-4703-bb42-babec053e499 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:18.344761 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10873914-fc81-4703-bb42-babec053e499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:18.357815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d3621-394a-4f18-b804-ffe8ed218bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:18.358958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d3621-394a-4f18-b804-ffe8ed218bff admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:18.359237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d3621-394a-4f18-b804-ffe8ed218bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:18.359581 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d3621-394a-4f18-b804-ffe8ed218bff admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:18.360862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d3621-394a-4f18-b804-ffe8ed218bff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:18.362980 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d3621-394a-4f18-b804-ffe8ed218bff admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:18.399043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-10873914-fc81-4703-bb42-babec053e499 None None] MFA Rules not processed for user `e8429576e286422cb09591897e82cd69`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:18.400779 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-10873914-fc81-4703-bb42-babec053e499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:18.464603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 958/1933] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:18 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:18.466038 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d3621-394a-4f18-b804-ffe8ed218bff admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:18.467919 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-831d3621-394a-4f18-b804-ffe8ed218bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:18.473757 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 976/1934] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:18.474679 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10873914-fc81-4703-bb42-babec053e499 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:18.480683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d3621-394a-4f18-b804-ffe8ed218bff admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:18.485577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebbfc70-b5ba-4974-8a63-201bbd18ba67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:18.486284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbfc70-b5ba-4974-8a63-201bbd18ba67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:18.486420 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10873914-fc81-4703-bb42-babec053e499 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:18.486699 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbfc70-b5ba-4974-8a63-201bbd18ba67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:18.487064 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbfc70-b5ba-4974-8a63-201bbd18ba67 admin admin] PATH_INFO: `/v3/projects/cbeac48c23bf40cf9a7132405f45b806` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:18.488258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebbfc70-b5ba-4974-8a63-201bbd18ba67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbeac48c23bf40cf9a7132405f45b806)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:18.490289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebbfc70-b5ba-4974-8a63-201bbd18ba67 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:18.491261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd65fd6-54ca-42a6-92c3-660a60f9e09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:18.492066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd65fd6-54ca-42a6-92c3-660a60f9e09b admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:18.496183 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd65fd6-54ca-42a6-92c3-660a60f9e09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:18.496183 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd65fd6-54ca-42a6-92c3-660a60f9e09b admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:18.496183 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd65fd6-54ca-42a6-92c3-660a60f9e09b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:18.496183 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd65fd6-54ca-42a6-92c3-660a60f9e09b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:18.506756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 977/1935] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:18.510514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd65fd6-54ca-42a6-92c3-660a60f9e09b admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:18.521555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd65fd6-54ca-42a6-92c3-660a60f9e09b admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:18.526264 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9192ac-29ed-42ff-ab32-068a20b19d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:18.526898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9192ac-29ed-42ff-ab32-068a20b19d06 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:18.527181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9192ac-29ed-42ff-ab32-068a20b19d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:18.527454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9192ac-29ed-42ff-ab32-068a20b19d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:18.528085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9192ac-29ed-42ff-ab32-068a20b19d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:18.529654 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9192ac-29ed-42ff-ab32-068a20b19d06 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:18.539712 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 978/1936] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:18.550476 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9192ac-29ed-42ff-ab32-068a20b19d06 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:18.559919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9192ac-29ed-42ff-ab32-068a20b19d06 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:18.564963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc95da-9d6a-4a9e-bab3-646e0de12648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:18.565580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc95da-9d6a-4a9e-bab3-646e0de12648 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:18.565861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc95da-9d6a-4a9e-bab3-646e0de12648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:18.566135 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc95da-9d6a-4a9e-bab3-646e0de12648 admin admin] PATH_INFO: `/v3/projects/ecf87234834642fb897cd7ad24ab3a19/users/783e805363ee444788177ee1359dac74/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:18.580750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc95da-9d6a-4a9e-bab3-646e0de12648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecf87234834642fb897cd7ad24ab3a19, user_id=783e805363ee444788177ee1359dac74, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:18.583509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc95da-9d6a-4a9e-bab3-646e0de12648 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:18.732819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 979/1937] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:18 2022] PUT /identity/v3/projects/ecf87234834642fb897cd7ad24ab3a19/users/783e805363ee444788177ee1359dac74/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:18.733841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc95da-9d6a-4a9e-bab3-646e0de12648 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:18.736445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-3efc95da-9d6a-4a9e-bab3-646e0de12648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:18.742471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9ebbfc70-b5ba-4974-8a63-201bbd18ba67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbeac48c23bf40cf9a7132405f45b806', 'request_id': 'req-9ebbfc70-b5ba-4974-8a63-201bbd18ba67'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:18.746603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9ebbfc70-b5ba-4974-8a63-201bbd18ba67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbeac48c23bf40cf9a7132405f45b806', 'request_id': 'req-9ebbfc70-b5ba-4974-8a63-201bbd18ba67'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:18.798949 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc95da-9d6a-4a9e-bab3-646e0de12648 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:18.845904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7edc2e-942a-4893-a3e8-5d5428064669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:18.846512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7edc2e-942a-4893-a3e8-5d5428064669 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:18.846939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7edc2e-942a-4893-a3e8-5d5428064669 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:18.847219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7edc2e-942a-4893-a3e8-5d5428064669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:18.857943 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9ebbfc70-b5ba-4974-8a63-201bbd18ba67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbeac48c23bf40cf9a7132405f45b806', 'request_id': 'req-9ebbfc70-b5ba-4974-8a63-201bbd18ba67'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:18.870460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9ebbfc70-b5ba-4974-8a63-201bbd18ba67 admin admin] The token cache is being invalidate because project cbeac48c23bf40cf9a7132405f45b806 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:18.870745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9ebbfc70-b5ba-4974-8a63-201bbd18ba67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:18.871418 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9ebbfc70-b5ba-4974-8a63-201bbd18ba67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:18.872893 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 959/1938] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:18 2022] DELETE /identity/v3/projects/cbeac48c23bf40cf9a7132405f45b806 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:18.877984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffeaa5-30e4-46b2-be29-995fbbf776f3 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:18.878271 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffeaa5-30e4-46b2-be29-995fbbf776f3 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:18.878546 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffeaa5-30e4-46b2-be29-995fbbf776f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:18.903097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7edc2e-942a-4893-a3e8-5d5428064669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:18.905422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7edc2e-942a-4893-a3e8-5d5428064669 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:18.906182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-83ffeaa5-30e4-46b2-be29-995fbbf776f3 None None] MFA Rules not processed for user `783e805363ee444788177ee1359dac74`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:18.907419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-83ffeaa5-30e4-46b2-be29-995fbbf776f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:18.932063 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 980/1939] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:18.934365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdf05a-ddc2-4322-abcf-db7b8f38f193 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:18.934665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdf05a-ddc2-4322-abcf-db7b8f38f193 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:18.934979 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdf05a-ddc2-4322-abcf-db7b8f38f193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:18.963489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-e4cdf05a-ddc2-4322-abcf-db7b8f38f193 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:18.965254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e4cdf05a-ddc2-4322-abcf-db7b8f38f193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:19.030212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 960/1940] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:19.036058 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffeaa5-30e4-46b2-be29-995fbbf776f3 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:19.039205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-83ffeaa5-30e4-46b2-be29-995fbbf776f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:19.070557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 981/1941] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:19.079634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc40af0-b0ee-4916-b31e-6a72935daaee None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:19.079927 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc40af0-b0ee-4916-b31e-6a72935daaee None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:19.080208 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc40af0-b0ee-4916-b31e-6a72935daaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:19.096932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:19.106867 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-7bc40af0-b0ee-4916-b31e-6a72935daaee None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:19.108863 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7bc40af0-b0ee-4916-b31e-6a72935daaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:19.125777 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 982/1942] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:19.126674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc40af0-b0ee-4916-b31e-6a72935daaee None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:19.138438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc40af0-b0ee-4916-b31e-6a72935daaee None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:19.143782 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738f868-67f5-457d-8687-c652993534f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:19.144469 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738f868-67f5-457d-8687-c652993534f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:19.144820 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738f868-67f5-457d-8687-c652993534f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:19.145101 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738f868-67f5-457d-8687-c652993534f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:19.145752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738f868-67f5-457d-8687-c652993534f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:36:19.146139 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738f868-67f5-457d-8687-c652993534f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:19.147707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738f868-67f5-457d-8687-c652993534f9 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:19.162176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 983/1943] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:36:19.164961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffeaa5-30e4-46b2-be29-995fbbf776f3 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:19.166324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738f868-67f5-457d-8687-c652993534f9 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:19.170165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e8011-b434-4a61-9461-032b9103869f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:19.171070 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e8011-b434-4a61-9461-032b9103869f admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:19.171419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e8011-b434-4a61-9461-032b9103869f admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:19.171765 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e8011-b434-4a61-9461-032b9103869f admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:19.173038 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e8011-b434-4a61-9461-032b9103869f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:19.175735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e8011-b434-4a61-9461-032b9103869f admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:19.177389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738f868-67f5-457d-8687-c652993534f9 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:19.183141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043e48c-8a78-4371-95e2-a3043be9b71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:19.184988 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043e48c-8a78-4371-95e2-a3043be9b71b admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:19.184988 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043e48c-8a78-4371-95e2-a3043be9b71b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:19.184988 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043e48c-8a78-4371-95e2-a3043be9b71b admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:19.184988 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043e48c-8a78-4371-95e2-a3043be9b71b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:19.187151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043e48c-8a78-4371-95e2-a3043be9b71b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:19.278142 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 984/1944] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:19 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:19.278721 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 961/1945] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:19 2022] POST /identity/v3/projects => generated 385 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:36:19.288596 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043e48c-8a78-4371-95e2-a3043be9b71b admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:19.291945 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e8011-b434-4a61-9461-032b9103869f admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:19.361354 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e8011-b434-4a61-9461-032b9103869f admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:19.366079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4d41f-27e1-4e0f-8585-386664aed7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:19.366691 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4d41f-27e1-4e0f-8585-386664aed7ed admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:19.367024 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4d41f-27e1-4e0f-8585-386664aed7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:19.367305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4d41f-27e1-4e0f-8585-386664aed7ed admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:19.367943 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4d41f-27e1-4e0f-8585-386664aed7ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:19.369297 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043e48c-8a78-4371-95e2-a3043be9b71b admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:19.369917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4d41f-27e1-4e0f-8585-386664aed7ed admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:19.374208 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e292d6-afb8-4923-b587-fdd2397e5ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:19.374838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e292d6-afb8-4923-b587-fdd2397e5ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:19.375120 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e292d6-afb8-4923-b587-fdd2397e5ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:19.375934 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e292d6-afb8-4923-b587-fdd2397e5ed9 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:19.376599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e292d6-afb8-4923-b587-fdd2397e5ed9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:19.378488 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e292d6-afb8-4923-b587-fdd2397e5ed9 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:19.486290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 985/1946] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:19 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:19.486757 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 962/1947] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:19 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:36:19.487463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e292d6-afb8-4923-b587-fdd2397e5ed9 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:19.489077 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-96e292d6-afb8-4923-b587-fdd2397e5ed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:19.490138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4d41f-27e1-4e0f-8585-386664aed7ed admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:19.499887 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4d41f-27e1-4e0f-8585-386664aed7ed admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:19.503777 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e292d6-afb8-4923-b587-fdd2397e5ed9 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:19.505227 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977806e2-3f16-4f11-b2e7-35bd947ecda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:19.505928 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977806e2-3f16-4f11-b2e7-35bd947ecda2 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:19.506292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977806e2-3f16-4f11-b2e7-35bd947ecda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:19.506635 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977806e2-3f16-4f11-b2e7-35bd947ecda2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:19.507387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977806e2-3f16-4f11-b2e7-35bd947ecda2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:19.509049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977806e2-3f16-4f11-b2e7-35bd947ecda2 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:19.510303 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67dd4c-6c0b-4d17-b3ba-9f60d93aed6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:19.512209 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67dd4c-6c0b-4d17-b3ba-9f60d93aed6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:19.512691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67dd4c-6c0b-4d17-b3ba-9f60d93aed6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:19.512907 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67dd4c-6c0b-4d17-b3ba-9f60d93aed6e admin admin] PATH_INFO: `/v3/users/8f2b43d09c034e07a7378f645a8004c9` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:19.514082 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67dd4c-6c0b-4d17-b3ba-9f60d93aed6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f2b43d09c034e07a7378f645a8004c9)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:19.516123 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67dd4c-6c0b-4d17-b3ba-9f60d93aed6e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:19.520182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 963/1948] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:19.521156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977806e2-3f16-4f11-b2e7-35bd947ecda2 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:19.531019 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977806e2-3f16-4f11-b2e7-35bd947ecda2 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:19.535975 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58decd7-b2b3-479a-ab6b-3f9b58a6252c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:19.536588 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58decd7-b2b3-479a-ab6b-3f9b58a6252c admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:19.536869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58decd7-b2b3-479a-ab6b-3f9b58a6252c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:19.537149 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58decd7-b2b3-479a-ab6b-3f9b58a6252c admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:19.537796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58decd7-b2b3-479a-ab6b-3f9b58a6252c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:19.539454 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58decd7-b2b3-479a-ab6b-3f9b58a6252c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:19.550639 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 964/1949] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:19.551491 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58decd7-b2b3-479a-ab6b-3f9b58a6252c admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:19.560726 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58decd7-b2b3-479a-ab6b-3f9b58a6252c admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:19.566200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67737ec3-5430-4063-b95b-0694eb67c219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:19.566802 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67737ec3-5430-4063-b95b-0694eb67c219 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:19.567168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67737ec3-5430-4063-b95b-0694eb67c219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:19.567447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67737ec3-5430-4063-b95b-0694eb67c219 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:19.568077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67737ec3-5430-4063-b95b-0694eb67c219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:19.569649 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67737ec3-5430-4063-b95b-0694eb67c219 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:19.579229 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 965/1950] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:19.579990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67737ec3-5430-4063-b95b-0694eb67c219 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:19.589274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67737ec3-5430-4063-b95b-0694eb67c219 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:19.594321 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d3c09-686d-43dd-ba65-5fb5185dbacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:19.594927 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d3c09-686d-43dd-ba65-5fb5185dbacd admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:19.595213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d3c09-686d-43dd-ba65-5fb5185dbacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:19.595490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d3c09-686d-43dd-ba65-5fb5185dbacd admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:19.596112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d3c09-686d-43dd-ba65-5fb5185dbacd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:19.597673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d3c09-686d-43dd-ba65-5fb5185dbacd admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:19.607398 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 966/1951] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:19.608146 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d3c09-686d-43dd-ba65-5fb5185dbacd admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:19.617782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d3c09-686d-43dd-ba65-5fb5185dbacd admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:19.622359 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8885d83-3949-4b7e-83f9-99eeefaf958c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:19.623010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8885d83-3949-4b7e-83f9-99eeefaf958c admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:19.623297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8885d83-3949-4b7e-83f9-99eeefaf958c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:19.623577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8885d83-3949-4b7e-83f9-99eeefaf958c admin admin] PATH_INFO: `/v3/projects/0fb9dfdcd1a44783a51f011176c00102/users/661c119077504138b1fba6f7bd9eca2b/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:19.638591 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8885d83-3949-4b7e-83f9-99eeefaf958c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fb9dfdcd1a44783a51f011176c00102, user_id=661c119077504138b1fba6f7bd9eca2b, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:19.641343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8885d83-3949-4b7e-83f9-99eeefaf958c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:19.799073 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 967/1952] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:19 2022] PUT /identity/v3/projects/0fb9dfdcd1a44783a51f011176c00102/users/661c119077504138b1fba6f7bd9eca2b/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:19.799073 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8885d83-3949-4b7e-83f9-99eeefaf958c admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:19.806251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ac67dd4c-6c0b-4d17-b3ba-9f60d93aed6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f2b43d09c034e07a7378f645a8004c9'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:19.861021 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8885d83-3949-4b7e-83f9-99eeefaf958c admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:19.865877 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1aa05-b385-408a-90db-11d5ad4ab5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:19.866548 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1aa05-b385-408a-90db-11d5ad4ab5b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:19.866861 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1aa05-b385-408a-90db-11d5ad4ab5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:19.867160 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1aa05-b385-408a-90db-11d5ad4ab5b6 admin admin] PATH_INFO: `/v3/projects/f5cd7ab190084a629de746b67ba1c164/users/bbc79451871a41b394b601db8b40f9d1/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:19.881611 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1aa05-b385-408a-90db-11d5ad4ab5b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5cd7ab190084a629de746b67ba1c164, user_id=bbc79451871a41b394b601db8b40f9d1, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:19.884606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1aa05-b385-408a-90db-11d5ad4ab5b6 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:19.936631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ac67dd4c-6c0b-4d17-b3ba-9f60d93aed6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f2b43d09c034e07a7378f645a8004c9'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:19.939299 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:19.951444 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ac67dd4c-6c0b-4d17-b3ba-9f60d93aed6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f2b43d09c034e07a7378f645a8004c9'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:19.952102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ac67dd4c-6c0b-4d17-b3ba-9f60d93aed6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f2b43d09c034e07a7378f645a8004c9'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:19.958851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ac67dd4c-6c0b-4d17-b3ba-9f60d93aed6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f2b43d09c034e07a7378f645a8004c9'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:19.964666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 986/1953] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:19 2022] DELETE /identity/v3/users/8f2b43d09c034e07a7378f645a8004c9 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:19.966133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e829ed-5f05-488f-904c-abb6d46c4dfb None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:19.966414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e829ed-5f05-488f-904c-abb6d46c4dfb None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:19.966687 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e829ed-5f05-488f-904c-abb6d46c4dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:19.978507 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 968/1954] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:19 2022] PUT /identity/v3/projects/f5cd7ab190084a629de746b67ba1c164/users/bbc79451871a41b394b601db8b40f9d1/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:19.980324 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63bd51-b046-48ee-8b57-4b8ee1b5ca51 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:19.980733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63bd51-b046-48ee-8b57-4b8ee1b5ca51 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:19.980973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63bd51-b046-48ee-8b57-4b8ee1b5ca51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:19.996942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-42e829ed-5f05-488f-904c-abb6d46c4dfb None None] MFA Rules not processed for user `661c119077504138b1fba6f7bd9eca2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:19.998226 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-42e829ed-5f05-488f-904c-abb6d46c4dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:20.009319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-dd63bd51-b046-48ee-8b57-4b8ee1b5ca51 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:20.010524 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-dd63bd51-b046-48ee-8b57-4b8ee1b5ca51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:20.070685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 987/1955] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:20.071901 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee5c88-629e-4929-986f-a9937aea2002 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:20.072241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee5c88-629e-4929-986f-a9937aea2002 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:20.072436 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee5c88-629e-4929-986f-a9937aea2002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:20.080280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 969/1956] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:20.081214 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63bd51-b046-48ee-8b57-4b8ee1b5ca51 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:20.082542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-dd63bd51-b046-48ee-8b57-4b8ee1b5ca51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:20.100805 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63bd51-b046-48ee-8b57-4b8ee1b5ca51 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:20.104178 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-15ee5c88-629e-4929-986f-a9937aea2002 None None] MFA Rules not processed for user `bbc79451871a41b394b601db8b40f9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:20.105372 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-15ee5c88-629e-4929-986f-a9937aea2002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:20.105584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e9dbd-380e-4646-85d8-10fb68bd272f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:20.106193 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e9dbd-380e-4646-85d8-10fb68bd272f admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:20.106475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e9dbd-380e-4646-85d8-10fb68bd272f admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:20.106746 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e9dbd-380e-4646-85d8-10fb68bd272f admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:20.107424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e9dbd-380e-4646-85d8-10fb68bd272f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:20.109384 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e9dbd-380e-4646-85d8-10fb68bd272f admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:20.168627 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 988/1957] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:20.169458 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee5c88-629e-4929-986f-a9937aea2002 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:20.170733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-15ee5c88-629e-4929-986f-a9937aea2002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:20.227240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 970/1958] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:20 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:20.227240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e9dbd-380e-4646-85d8-10fb68bd272f admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:20.227240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-156e9dbd-380e-4646-85d8-10fb68bd272f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:20.227671 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee5c88-629e-4929-986f-a9937aea2002 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:20.276593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103ec07-aa90-42cd-bdee-b247973d089f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:20.277136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103ec07-aa90-42cd-bdee-b247973d089f service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:20.277423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103ec07-aa90-42cd-bdee-b247973d089f service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:20.277831 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103ec07-aa90-42cd-bdee-b247973d089f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:20.292286 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e9dbd-380e-4646-85d8-10fb68bd272f admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:20.297035 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310be42-60e0-4c1e-a01a-815d2c65e56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:20.298133 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310be42-60e0-4c1e-a01a-815d2c65e56e admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:20.298409 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310be42-60e0-4c1e-a01a-815d2c65e56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:20.298678 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310be42-60e0-4c1e-a01a-815d2c65e56e admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:20.299332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310be42-60e0-4c1e-a01a-815d2c65e56e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:20.301466 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310be42-60e0-4c1e-a01a-815d2c65e56e admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:20.341099 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103ec07-aa90-42cd-bdee-b247973d089f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:20.342553 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103ec07-aa90-42cd-bdee-b247973d089f service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:20.352152 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 989/1959] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:20.352935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103ec07-aa90-42cd-bdee-b247973d089f service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:20.362819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103ec07-aa90-42cd-bdee-b247973d089f service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:20.367657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644daddd-1978-4b70-ab83-18b5002b32e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:20.368227 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644daddd-1978-4b70-ab83-18b5002b32e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:20.368514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644daddd-1978-4b70-ab83-18b5002b32e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:20.368792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644daddd-1978-4b70-ab83-18b5002b32e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:20.369459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644daddd-1978-4b70-ab83-18b5002b32e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:20.371438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644daddd-1978-4b70-ab83-18b5002b32e3 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:20.427518 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 971/1960] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:20 2022] POST /identity/v3/projects => generated 383 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:20.428527 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310be42-60e0-4c1e-a01a-815d2c65e56e admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:20.458234 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 990/1961] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:20 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:20.461337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644daddd-1978-4b70-ab83-18b5002b32e3 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:20.472938 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644daddd-1978-4b70-ab83-18b5002b32e3 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:20.496975 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310be42-60e0-4c1e-a01a-815d2c65e56e admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:20.503461 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddc6d3-181a-433f-af90-40f001cfc2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:20.504302 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddc6d3-181a-433f-af90-40f001cfc2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:20.504588 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddc6d3-181a-433f-af90-40f001cfc2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:20.505365 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddc6d3-181a-433f-af90-40f001cfc2d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:20.506205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddc6d3-181a-433f-af90-40f001cfc2d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:20.508314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddc6d3-181a-433f-af90-40f001cfc2d2 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:20.535836 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729f567-e5c4-4dc6-8289-ee38a19e3fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:20.536380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729f567-e5c4-4dc6-8289-ee38a19e3fde admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:20.536662 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729f567-e5c4-4dc6-8289-ee38a19e3fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:20.536927 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729f567-e5c4-4dc6-8289-ee38a19e3fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:20.537572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729f567-e5c4-4dc6-8289-ee38a19e3fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:20.539198 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729f567-e5c4-4dc6-8289-ee38a19e3fde admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:20.553142 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 991/1962] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:20.554450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729f567-e5c4-4dc6-8289-ee38a19e3fde admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:20.565123 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729f567-e5c4-4dc6-8289-ee38a19e3fde admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:20.570059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e1637-8f58-409f-903c-f32c32575e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:20.570578 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e1637-8f58-409f-903c-f32c32575e56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:20.570910 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e1637-8f58-409f-903c-f32c32575e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:20.571190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e1637-8f58-409f-903c-f32c32575e56 admin admin] PATH_INFO: `/v3/projects/57da5873d3a546e4a678ebd89c5c62f2/users/2dbc40fd2d2f45b89e0eef050bcd612a/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:20.586040 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e1637-8f58-409f-903c-f32c32575e56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57da5873d3a546e4a678ebd89c5c62f2, user_id=2dbc40fd2d2f45b89e0eef050bcd612a, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:20.588771 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e1637-8f58-409f-903c-f32c32575e56 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:20.631341 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 972/1963] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:20 2022] POST /identity/v3/users => generated 298 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:36:20.637103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddc6d3-181a-433f-af90-40f001cfc2d2 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:20.647054 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddc6d3-181a-433f-af90-40f001cfc2d2 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:20.651939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edb2eb-0006-4989-aa93-96ff8420980a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:20.652616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edb2eb-0006-4989-aa93-96ff8420980a admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:20.653035 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edb2eb-0006-4989-aa93-96ff8420980a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:20.653376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edb2eb-0006-4989-aa93-96ff8420980a admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:20.654114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edb2eb-0006-4989-aa93-96ff8420980a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:20.656277 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edb2eb-0006-4989-aa93-96ff8420980a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:20.667089 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 973/1964] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:20.671127 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edb2eb-0006-4989-aa93-96ff8420980a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:20.680805 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edb2eb-0006-4989-aa93-96ff8420980a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:20.685535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323ef50-58f8-4f75-8649-0ec7727788b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:20.686129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323ef50-58f8-4f75-8649-0ec7727788b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:20.686486 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323ef50-58f8-4f75-8649-0ec7727788b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:20.686668 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323ef50-58f8-4f75-8649-0ec7727788b7 admin admin] PATH_INFO: `/v3/projects/6f75ae3652a84a188b33fb15a6f0bad8/users/4ff900a6e727483885a4496e86d065ec/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:20.701645 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323ef50-58f8-4f75-8649-0ec7727788b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f75ae3652a84a188b33fb15a6f0bad8, user_id=4ff900a6e727483885a4496e86d065ec, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:20.704616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323ef50-58f8-4f75-8649-0ec7727788b7 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:20.744769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 992/1965] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:20 2022] PUT /identity/v3/projects/57da5873d3a546e4a678ebd89c5c62f2/users/2dbc40fd2d2f45b89e0eef050bcd612a/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:20.748645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e1637-8f58-409f-903c-f32c32575e56 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:20.784490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:20.810354 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e1637-8f58-409f-903c-f32c32575e56 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:20.815101 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd6d1a-3322-4537-bf5b-f6d26f166376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:20.815721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd6d1a-3322-4537-bf5b-f6d26f166376 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:20.815992 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd6d1a-3322-4537-bf5b-f6d26f166376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:20.816352 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd6d1a-3322-4537-bf5b-f6d26f166376 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:20.816991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd6d1a-3322-4537-bf5b-f6d26f166376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:20.818585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd6d1a-3322-4537-bf5b-f6d26f166376 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:20.840385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 993/1966] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:20.840385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd6d1a-3322-4537-bf5b-f6d26f166376 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:20.847439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd6d1a-3322-4537-bf5b-f6d26f166376 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:20.853111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b276f01-ead7-4172-9527-918d00c6d959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:20.853784 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b276f01-ead7-4172-9527-918d00c6d959 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:20.854085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b276f01-ead7-4172-9527-918d00c6d959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:20.854498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b276f01-ead7-4172-9527-918d00c6d959 admin admin] PATH_INFO: `/v3/projects/57da5873d3a546e4a678ebd89c5c62f2/users/2dbc40fd2d2f45b89e0eef050bcd612a/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:20.856343 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b276f01-ead7-4172-9527-918d00c6d959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57da5873d3a546e4a678ebd89c5c62f2, user_id=2dbc40fd2d2f45b89e0eef050bcd612a, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:20.860473 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b276f01-ead7-4172-9527-918d00c6d959 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:20.875495 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 994/1967] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:20 2022] PUT /identity/v3/projects/57da5873d3a546e4a678ebd89c5c62f2/users/2dbc40fd2d2f45b89e0eef050bcd612a/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:20.879692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 974/1968] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:20 2022] PUT /identity/v3/projects/6f75ae3652a84a188b33fb15a6f0bad8/users/4ff900a6e727483885a4496e86d065ec/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:20.888002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b276f01-ead7-4172-9527-918d00c6d959 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:20.890697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fb71d-be62-45f1-9dd0-cf492c9a2819 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:20.891795 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fb71d-be62-45f1-9dd0-cf492c9a2819 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:20.892011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fb71d-be62-45f1-9dd0-cf492c9a2819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:20.934361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-053fb71d-be62-45f1-9dd0-cf492c9a2819 None None] MFA Rules not processed for user `2dbc40fd2d2f45b89e0eef050bcd612a`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:20.936224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-053fb71d-be62-45f1-9dd0-cf492c9a2819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:20.965928 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b276f01-ead7-4172-9527-918d00c6d959 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:20.968874 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:20.974323 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda0a13-f971-4cc1-8130-7f60ebd41f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:20.975189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda0a13-f971-4cc1-8130-7f60ebd41f61 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:20.975546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda0a13-f971-4cc1-8130-7f60ebd41f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:20.975900 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda0a13-f971-4cc1-8130-7f60ebd41f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:20.976710 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda0a13-f971-4cc1-8130-7f60ebd41f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:20.978475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda0a13-f971-4cc1-8130-7f60ebd41f61 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:20.990971 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 995/1969] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:20.994465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda0a13-f971-4cc1-8130-7f60ebd41f61 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:21.007363 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda0a13-f971-4cc1-8130-7f60ebd41f61 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:21.012336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace0bca-5830-4268-99dd-bc68e92599e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:21.013008 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace0bca-5830-4268-99dd-bc68e92599e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:21.013295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace0bca-5830-4268-99dd-bc68e92599e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:21.013562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace0bca-5830-4268-99dd-bc68e92599e9 admin admin] PATH_INFO: `/v3/projects/6f75ae3652a84a188b33fb15a6f0bad8/users/4ff900a6e727483885a4496e86d065ec/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:21.015540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace0bca-5830-4268-99dd-bc68e92599e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f75ae3652a84a188b33fb15a6f0bad8, user_id=4ff900a6e727483885a4496e86d065ec, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:21.019400 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace0bca-5830-4268-99dd-bc68e92599e9 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:21.034234 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 996/1970] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:20 2022] PUT /identity/v3/projects/6f75ae3652a84a188b33fb15a6f0bad8/users/4ff900a6e727483885a4496e86d065ec/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:21.036250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 975/1971] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:21.044374 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834991b-9a64-4d6d-a3cb-34caa0f977e4 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:21.045157 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834991b-9a64-4d6d-a3cb-34caa0f977e4 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:21.045157 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834991b-9a64-4d6d-a3cb-34caa0f977e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:21.055448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fb71d-be62-45f1-9dd0-cf492c9a2819 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:21.057128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-053fb71d-be62-45f1-9dd0-cf492c9a2819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:21.075554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-4834991b-9a64-4d6d-a3cb-34caa0f977e4 None None] MFA Rules not processed for user `4ff900a6e727483885a4496e86d065ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:21.077011 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-4834991b-9a64-4d6d-a3cb-34caa0f977e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:21.127997 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fb71d-be62-45f1-9dd0-cf492c9a2819 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:21.162941 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 997/1972] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:21.176389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834991b-9a64-4d6d-a3cb-34caa0f977e4 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:21.176389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-4834991b-9a64-4d6d-a3cb-34caa0f977e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:21.232733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc27b72-7bc2-40cb-a171-4a14c942d55c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:21.236269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc27b72-7bc2-40cb-a171-4a14c942d55c service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:21.236588 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc27b72-7bc2-40cb-a171-4a14c942d55c service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:21.236875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc27b72-7bc2-40cb-a171-4a14c942d55c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:21.263107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834991b-9a64-4d6d-a3cb-34caa0f977e4 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:21.271259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d31198-461b-423e-9394-d74c7a53233a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:21.272887 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d31198-461b-423e-9394-d74c7a53233a service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:21.273618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d31198-461b-423e-9394-d74c7a53233a service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:21.274150 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d31198-461b-423e-9394-d74c7a53233a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:21.277354 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-57d31198-461b-423e-9394-d74c7a53233a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:21.347133 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc27b72-7bc2-40cb-a171-4a14c942d55c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:21.348985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc27b72-7bc2-40cb-a171-4a14c942d55c service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:21.361307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 976/1973] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:21.362257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc27b72-7bc2-40cb-a171-4a14c942d55c service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:21.363768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-fcc27b72-7bc2-40cb-a171-4a14c942d55c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:21.379896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d31198-461b-423e-9394-d74c7a53233a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:21.381413 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d31198-461b-423e-9394-d74c7a53233a service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:21.394107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 998/1974] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:36:21.410622 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d31198-461b-423e-9394-d74c7a53233a service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:21.455005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc27b72-7bc2-40cb-a171-4a14c942d55c service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:21.461686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add6b21-fc4a-4196-8e15-64993a845ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:21.462017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add6b21-fc4a-4196-8e15-64993a845ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:21.462320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add6b21-fc4a-4196-8e15-64993a845ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:21.462570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add6b21-fc4a-4196-8e15-64993a845ca0 admin admin] PATH_INFO: `/v3/projects/0c9f2cb345994547b81ea276771536cc` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:21.464211 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add6b21-fc4a-4196-8e15-64993a845ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c9f2cb345994547b81ea276771536cc)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:21.467231 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add6b21-fc4a-4196-8e15-64993a845ca0 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:21.487252 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d31198-461b-423e-9394-d74c7a53233a service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:21.493512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa7886-fbce-43c3-91b1-b3c2fe6c5e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:21.494208 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa7886-fbce-43c3-91b1-b3c2fe6c5e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:21.494558 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa7886-fbce-43c3-91b1-b3c2fe6c5e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:21.494933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa7886-fbce-43c3-91b1-b3c2fe6c5e8c admin admin] PATH_INFO: `/v3/users/661c119077504138b1fba6f7bd9eca2b` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:21.496167 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa7886-fbce-43c3-91b1-b3c2fe6c5e8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=661c119077504138b1fba6f7bd9eca2b)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:21.498141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa7886-fbce-43c3-91b1-b3c2fe6c5e8c admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:21.607378 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8add6b21-fc4a-4196-8e15-64993a845ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c9f2cb345994547b81ea276771536cc', 'request_id': 'req-8add6b21-fc4a-4196-8e15-64993a845ca0'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:21.613023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8add6b21-fc4a-4196-8e15-64993a845ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c9f2cb345994547b81ea276771536cc', 'request_id': 'req-8add6b21-fc4a-4196-8e15-64993a845ca0'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:21.749537 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8add6b21-fc4a-4196-8e15-64993a845ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c9f2cb345994547b81ea276771536cc', 'request_id': 'req-8add6b21-fc4a-4196-8e15-64993a845ca0'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:21.762818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8add6b21-fc4a-4196-8e15-64993a845ca0 admin admin] The token cache is being invalidate because project 0c9f2cb345994547b81ea276771536cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:21.763431 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8add6b21-fc4a-4196-8e15-64993a845ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:21.763918 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8add6b21-fc4a-4196-8e15-64993a845ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:21.768716 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 977/1975] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:21 2022] DELETE /identity/v3/projects/0c9f2cb345994547b81ea276771536cc => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:21.769734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add6b21-fc4a-4196-8e15-64993a845ca0 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:21.771454 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8add6b21-fc4a-4196-8e15-64993a845ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:21.775832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0aaa7886-fbce-43c3-91b1-b3c2fe6c5e8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '661c119077504138b1fba6f7bd9eca2b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:21.839789 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add6b21-fc4a-4196-8e15-64993a845ca0 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:21.846640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d544fd4-df11-47f7-91e4-734be8f01ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:21.847322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544fd4-df11-47f7-91e4-734be8f01ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:21.847604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544fd4-df11-47f7-91e4-734be8f01ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:21.847874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544fd4-df11-47f7-91e4-734be8f01ae4 admin admin] PATH_INFO: `/v3/users/11c431818ade47ebb256f33011e75c2c` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:21.849044 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d544fd4-df11-47f7-91e4-734be8f01ae4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11c431818ade47ebb256f33011e75c2c)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:21.851090 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d544fd4-df11-47f7-91e4-734be8f01ae4 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:21.868463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0aaa7886-fbce-43c3-91b1-b3c2fe6c5e8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '661c119077504138b1fba6f7bd9eca2b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:21.882284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0aaa7886-fbce-43c3-91b1-b3c2fe6c5e8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '661c119077504138b1fba6f7bd9eca2b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:21.884345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0aaa7886-fbce-43c3-91b1-b3c2fe6c5e8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '661c119077504138b1fba6f7bd9eca2b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:21.898861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0aaa7886-fbce-43c3-91b1-b3c2fe6c5e8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '661c119077504138b1fba6f7bd9eca2b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:21.912744 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 999/1976] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:21 2022] DELETE /identity/v3/users/661c119077504138b1fba6f7bd9eca2b => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:21.923896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa1275-2bf9-4345-85cf-33ca3bbfb608 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:21.924168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa1275-2bf9-4345-85cf-33ca3bbfb608 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:21.924438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa1275-2bf9-4345-85cf-33ca3bbfb608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:21.971010 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-39fa1275-2bf9-4345-85cf-33ca3bbfb608 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:21.972254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-39fa1275-2bf9-4345-85cf-33ca3bbfb608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:22.037081 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1000/1977] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:22.038919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534e6ae-2c6d-4752-8640-d289dcebb4c8 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:22.038919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534e6ae-2c6d-4752-8640-d289dcebb4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:22.038919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534e6ae-2c6d-4752-8640-d289dcebb4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:22.058439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-7534e6ae-2c6d-4752-8640-d289dcebb4c8 None None] MFA Rules not processed for user `e8429576e286422cb09591897e82cd69`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:22.059748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7534e6ae-2c6d-4752-8640-d289dcebb4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:22.104976 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9d544fd4-df11-47f7-91e4-734be8f01ae4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11c431818ade47ebb256f33011e75c2c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:22.124705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1001/1978] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:22.125512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534e6ae-2c6d-4752-8640-d289dcebb4c8 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:22.126815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7534e6ae-2c6d-4752-8640-d289dcebb4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:22.210827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534e6ae-2c6d-4752-8640-d289dcebb4c8 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:22.214437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d739da-d0a9-42f2-ac97-c4bd73e1cf18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:22.215350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d739da-d0a9-42f2-ac97-c4bd73e1cf18 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:22.215990 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d739da-d0a9-42f2-ac97-c4bd73e1cf18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:22.215990 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d739da-d0a9-42f2-ac97-c4bd73e1cf18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:22.226154 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d739da-d0a9-42f2-ac97-c4bd73e1cf18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:22.227859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d739da-d0a9-42f2-ac97-c4bd73e1cf18 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:22.229276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9d544fd4-df11-47f7-91e4-734be8f01ae4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11c431818ade47ebb256f33011e75c2c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:22.230143 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9d544fd4-df11-47f7-91e4-734be8f01ae4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11c431818ade47ebb256f33011e75c2c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:22.238623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1002/1979] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:22.243992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9d544fd4-df11-47f7-91e4-734be8f01ae4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11c431818ade47ebb256f33011e75c2c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:22.250956 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9d544fd4-df11-47f7-91e4-734be8f01ae4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11c431818ade47ebb256f33011e75c2c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:22.258447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 978/1980] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:21 2022] DELETE /identity/v3/users/11c431818ade47ebb256f33011e75c2c => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:22.258754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:22.709955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:23.019054 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d739da-d0a9-42f2-ac97-c4bd73e1cf18 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:23.020643 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-48d739da-d0a9-42f2-ac97-c4bd73e1cf18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:23.034984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d739da-d0a9-42f2-ac97-c4bd73e1cf18 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:23.134371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67db2ff-00d8-48c3-afef-233d8d9b6b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:23.135441 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67db2ff-00d8-48c3-afef-233d8d9b6b52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:23.135879 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67db2ff-00d8-48c3-afef-233d8d9b6b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:23.137098 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67db2ff-00d8-48c3-afef-233d8d9b6b52 admin admin] PATH_INFO: `/v3/projects/41d27f1905194b70abb6d0b2ea922d8a` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:23.138885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67db2ff-00d8-48c3-afef-233d8d9b6b52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41d27f1905194b70abb6d0b2ea922d8a)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:23.143698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67db2ff-00d8-48c3-afef-233d8d9b6b52 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:23.168026 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d544fd4-df11-47f7-91e4-734be8f01ae4 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:23.168026 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9d544fd4-df11-47f7-91e4-734be8f01ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:23.186337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d544fd4-df11-47f7-91e4-734be8f01ae4 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:23.192757 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb882ad1-5644-4598-bb34-eb737c11b613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:23.195052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb882ad1-5644-4598-bb34-eb737c11b613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:23.195661 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb882ad1-5644-4598-bb34-eb737c11b613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:23.196112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb882ad1-5644-4598-bb34-eb737c11b613 admin admin] PATH_INFO: `/v3/projects/0fb9dfdcd1a44783a51f011176c00102` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:23.198032 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb882ad1-5644-4598-bb34-eb737c11b613 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fb9dfdcd1a44783a51f011176c00102)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:23.201412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb882ad1-5644-4598-bb34-eb737c11b613 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:23.391296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b67db2ff-00d8-48c3-afef-233d8d9b6b52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41d27f1905194b70abb6d0b2ea922d8a', 'request_id': 'req-b67db2ff-00d8-48c3-afef-233d8d9b6b52'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:23.395635 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-cb882ad1-5644-4598-bb34-eb737c11b613 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fb9dfdcd1a44783a51f011176c00102', 'request_id': 'req-cb882ad1-5644-4598-bb34-eb737c11b613'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:23.399344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b67db2ff-00d8-48c3-afef-233d8d9b6b52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41d27f1905194b70abb6d0b2ea922d8a', 'request_id': 'req-b67db2ff-00d8-48c3-afef-233d8d9b6b52'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:23.402495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-cb882ad1-5644-4598-bb34-eb737c11b613 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fb9dfdcd1a44783a51f011176c00102', 'request_id': 'req-cb882ad1-5644-4598-bb34-eb737c11b613'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:23.549933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b67db2ff-00d8-48c3-afef-233d8d9b6b52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41d27f1905194b70abb6d0b2ea922d8a', 'request_id': 'req-b67db2ff-00d8-48c3-afef-233d8d9b6b52'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:23.562438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b67db2ff-00d8-48c3-afef-233d8d9b6b52 admin admin] The token cache is being invalidate because project 41d27f1905194b70abb6d0b2ea922d8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:23.563371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b67db2ff-00d8-48c3-afef-233d8d9b6b52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:23.564066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b67db2ff-00d8-48c3-afef-233d8d9b6b52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:23.565692 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1003/1981] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:23 2022] DELETE /identity/v3/projects/41d27f1905194b70abb6d0b2ea922d8a => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:23.570203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c0455-8248-41ea-91c8-4c77673c6fb5 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:23.570489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c0455-8248-41ea-91c8-4c77673c6fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:23.570836 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c0455-8248-41ea-91c8-4c77673c6fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:23.590883 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-493c0455-8248-41ea-91c8-4c77673c6fb5 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:23.592098 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-493c0455-8248-41ea-91c8-4c77673c6fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:23.627586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-cb882ad1-5644-4598-bb34-eb737c11b613 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fb9dfdcd1a44783a51f011176c00102', 'request_id': 'req-cb882ad1-5644-4598-bb34-eb737c11b613'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:23.640573 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-cb882ad1-5644-4598-bb34-eb737c11b613 admin admin] The token cache is being invalidate because project 0fb9dfdcd1a44783a51f011176c00102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:23.640971 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-cb882ad1-5644-4598-bb34-eb737c11b613 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:23.642219 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-cb882ad1-5644-4598-bb34-eb737c11b613 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:23.643835 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 979/1982] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:23 2022] DELETE /identity/v3/projects/0fb9dfdcd1a44783a51f011176c00102 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:23.647716 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb882ad1-5644-4598-bb34-eb737c11b613 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:23.649036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-cb882ad1-5644-4598-bb34-eb737c11b613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:23.652030 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1004/1983] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:23.659952 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc61b9c-e8f7-4cb5-b0af-12130eb44981 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:23.660292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc61b9c-e8f7-4cb5-b0af-12130eb44981 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:23.660571 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc61b9c-e8f7-4cb5-b0af-12130eb44981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:23.662368 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb882ad1-5644-4598-bb34-eb737c11b613 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:23.667310 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab950e4-a2c6-4b7e-9c98-84fb9353d58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:23.668000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab950e4-a2c6-4b7e-9c98-84fb9353d58c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:23.668288 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab950e4-a2c6-4b7e-9c98-84fb9353d58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:23.668560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab950e4-a2c6-4b7e-9c98-84fb9353d58c admin admin] PATH_INFO: `/v3/users/2dbc40fd2d2f45b89e0eef050bcd612a` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:23.669814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab950e4-a2c6-4b7e-9c98-84fb9353d58c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dbc40fd2d2f45b89e0eef050bcd612a)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:23.671774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab950e4-a2c6-4b7e-9c98-84fb9353d58c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:23.682062 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-edc61b9c-e8f7-4cb5-b0af-12130eb44981 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:23.683351 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-edc61b9c-e8f7-4cb5-b0af-12130eb44981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:23.691957 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1005/1984] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:23.696539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc61b9c-e8f7-4cb5-b0af-12130eb44981 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:23.706082 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc61b9c-e8f7-4cb5-b0af-12130eb44981 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:23.710715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450914d7-1ffb-4814-bfa6-4384e36bc6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:23.711338 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450914d7-1ffb-4814-bfa6-4384e36bc6e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:23.711617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450914d7-1ffb-4814-bfa6-4384e36bc6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:23.711884 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450914d7-1ffb-4814-bfa6-4384e36bc6e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:23.712517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450914d7-1ffb-4814-bfa6-4384e36bc6e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:36:23.712811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450914d7-1ffb-4814-bfa6-4384e36bc6e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:23.714476 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450914d7-1ffb-4814-bfa6-4384e36bc6e2 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:23.729859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1006/1985] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:36:23.731252 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450914d7-1ffb-4814-bfa6-4384e36bc6e2 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:23.741206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450914d7-1ffb-4814-bfa6-4384e36bc6e2 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:23.746382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60687abe-ae68-44ab-a5db-4e31e184a78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:23.747032 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60687abe-ae68-44ab-a5db-4e31e184a78e admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:23.747312 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60687abe-ae68-44ab-a5db-4e31e184a78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:23.747580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60687abe-ae68-44ab-a5db-4e31e184a78e admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:23.748577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60687abe-ae68-44ab-a5db-4e31e184a78e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:23.751711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60687abe-ae68-44ab-a5db-4e31e184a78e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:23.885256 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-dab950e4-a2c6-4b7e-9c98-84fb9353d58c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dbc40fd2d2f45b89e0eef050bcd612a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:23.886299 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1007/1986] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:23 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:23.890618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60687abe-ae68-44ab-a5db-4e31e184a78e admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:23.955181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60687abe-ae68-44ab-a5db-4e31e184a78e admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:23.959976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96fb3a-9f61-491c-9509-f57c318a2752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:23.960584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96fb3a-9f61-491c-9509-f57c318a2752 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:23.960906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96fb3a-9f61-491c-9509-f57c318a2752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:23.961184 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96fb3a-9f61-491c-9509-f57c318a2752 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:23.961812 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96fb3a-9f61-491c-9509-f57c318a2752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:23.963762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96fb3a-9f61-491c-9509-f57c318a2752 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:23.974167 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-dab950e4-a2c6-4b7e-9c98-84fb9353d58c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dbc40fd2d2f45b89e0eef050bcd612a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:23.975192 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-dab950e4-a2c6-4b7e-9c98-84fb9353d58c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dbc40fd2d2f45b89e0eef050bcd612a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:23.988744 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-dab950e4-a2c6-4b7e-9c98-84fb9353d58c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dbc40fd2d2f45b89e0eef050bcd612a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:23.995653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-dab950e4-a2c6-4b7e-9c98-84fb9353d58c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dbc40fd2d2f45b89e0eef050bcd612a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:24.001563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 980/1987] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:23 2022] DELETE /identity/v3/users/2dbc40fd2d2f45b89e0eef050bcd612a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:24.099555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1008/1988] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:23 2022] POST /identity/v3/users => generated 287 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:36:24.105212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab950e4-a2c6-4b7e-9c98-84fb9353d58c admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:24.106619 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-dab950e4-a2c6-4b7e-9c98-84fb9353d58c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:24.122106 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:24.188253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab950e4-a2c6-4b7e-9c98-84fb9353d58c admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:24.193188 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74e95f-332e-44dd-8965-80b85eaecd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:24.193807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74e95f-332e-44dd-8965-80b85eaecd65 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:24.194091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74e95f-332e-44dd-8965-80b85eaecd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:24.194364 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74e95f-332e-44dd-8965-80b85eaecd65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:24.195052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74e95f-332e-44dd-8965-80b85eaecd65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:24.196895 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74e95f-332e-44dd-8965-80b85eaecd65 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:24.208144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 981/1989] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:24.212266 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96fb3a-9f61-491c-9509-f57c318a2752 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:24.224760 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96fb3a-9f61-491c-9509-f57c318a2752 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:24.233931 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97abbc3-146b-4a12-ba69-40a8e4e2ec32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:24.234568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97abbc3-146b-4a12-ba69-40a8e4e2ec32 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:24.234933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97abbc3-146b-4a12-ba69-40a8e4e2ec32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:24.235394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97abbc3-146b-4a12-ba69-40a8e4e2ec32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:24.235860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97abbc3-146b-4a12-ba69-40a8e4e2ec32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:24.237638 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97abbc3-146b-4a12-ba69-40a8e4e2ec32 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:24.255578 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1009/1990] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:24.257138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74e95f-332e-44dd-8965-80b85eaecd65 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:24.282396 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74e95f-332e-44dd-8965-80b85eaecd65 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:24.288778 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20c20d-ce3c-4d3c-851e-e059c65987e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:24.289617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20c20d-ce3c-4d3c-851e-e059c65987e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:24.289904 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20c20d-ce3c-4d3c-851e-e059c65987e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:24.290359 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20c20d-ce3c-4d3c-851e-e059c65987e5 admin admin] PATH_INFO: `/v3/projects/ada19ac8bad54797b8b43350164d3eee/users/ba2dd9647d94417d800daa14ee56fdef/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:24.294559 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:24.315337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20c20d-ce3c-4d3c-851e-e059c65987e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ada19ac8bad54797b8b43350164d3eee, user_id=ba2dd9647d94417d800daa14ee56fdef, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:24.318291 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20c20d-ce3c-4d3c-851e-e059c65987e5 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:24.401880 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 982/1991] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:24 2022] PUT /identity/v3/projects/ada19ac8bad54797b8b43350164d3eee/users/ba2dd9647d94417d800daa14ee56fdef/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:24.414111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e517b5-9ddd-49fc-b661-08dc23c9f0c6 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:24.414456 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e517b5-9ddd-49fc-b661-08dc23c9f0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:24.414840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e517b5-9ddd-49fc-b661-08dc23c9f0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:24.458259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-48e517b5-9ddd-49fc-b661-08dc23c9f0c6 None None] MFA Rules not processed for user `ba2dd9647d94417d800daa14ee56fdef`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:24.459534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-48e517b5-9ddd-49fc-b661-08dc23c9f0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:24.530461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1010/1992] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:24.535534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20c20d-ce3c-4d3c-851e-e059c65987e5 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:24.553845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e517b5-9ddd-49fc-b661-08dc23c9f0c6 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:24.556545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-48e517b5-9ddd-49fc-b661-08dc23c9f0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:24.612517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20c20d-ce3c-4d3c-851e-e059c65987e5 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:24.617312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff95618-191e-4540-85c3-fb9a42a1dfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:24.618147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff95618-191e-4540-85c3-fb9a42a1dfa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:24.618379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff95618-191e-4540-85c3-fb9a42a1dfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:24.618666 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff95618-191e-4540-85c3-fb9a42a1dfa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:24.619614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff95618-191e-4540-85c3-fb9a42a1dfa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:24.621518 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff95618-191e-4540-85c3-fb9a42a1dfa0 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:24.633490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e517b5-9ddd-49fc-b661-08dc23c9f0c6 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:24.640503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6ed7f-8715-44c2-8afa-fafec679e690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:24.641800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6ed7f-8715-44c2-8afa-fafec679e690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:24.642228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6ed7f-8715-44c2-8afa-fafec679e690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:24.642511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6ed7f-8715-44c2-8afa-fafec679e690 admin admin] PATH_INFO: `/v3/projects/57da5873d3a546e4a678ebd89c5c62f2` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:24.644777 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6ed7f-8715-44c2-8afa-fafec679e690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57da5873d3a546e4a678ebd89c5c62f2)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:24.646742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6ed7f-8715-44c2-8afa-fafec679e690 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:24.781068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 983/1993] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:24 2022] POST /identity/v3/projects => generated 371 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:36:24.781068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff95618-191e-4540-85c3-fb9a42a1dfa0 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:24.783498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-6ff95618-191e-4540-85c3-fb9a42a1dfa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:24.792100 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8ec6ed7f-8715-44c2-8afa-fafec679e690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57da5873d3a546e4a678ebd89c5c62f2', 'request_id': 'req-8ec6ed7f-8715-44c2-8afa-fafec679e690'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:24.799439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8ec6ed7f-8715-44c2-8afa-fafec679e690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57da5873d3a546e4a678ebd89c5c62f2', 'request_id': 'req-8ec6ed7f-8715-44c2-8afa-fafec679e690'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:24.839857 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff95618-191e-4540-85c3-fb9a42a1dfa0 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:24.843753 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cc123-0487-4de0-b1de-b81ea7540c67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:24.844977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cc123-0487-4de0-b1de-b81ea7540c67 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:24.845256 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cc123-0487-4de0-b1de-b81ea7540c67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:24.845526 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cc123-0487-4de0-b1de-b81ea7540c67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:24.847283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e63cc123-0487-4de0-b1de-b81ea7540c67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:24.903694 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cc123-0487-4de0-b1de-b81ea7540c67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:24.905067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cc123-0487-4de0-b1de-b81ea7540c67 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:24.914543 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 984/1994] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:24.915339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cc123-0487-4de0-b1de-b81ea7540c67 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:24.919073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8ec6ed7f-8715-44c2-8afa-fafec679e690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57da5873d3a546e4a678ebd89c5c62f2', 'request_id': 'req-8ec6ed7f-8715-44c2-8afa-fafec679e690'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:24.933136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8ec6ed7f-8715-44c2-8afa-fafec679e690 admin admin] The token cache is being invalidate because project 57da5873d3a546e4a678ebd89c5c62f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:24.933439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8ec6ed7f-8715-44c2-8afa-fafec679e690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:24.934111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8ec6ed7f-8715-44c2-8afa-fafec679e690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:24.935675 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1011/1995] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:24 2022] DELETE /identity/v3/projects/57da5873d3a546e4a678ebd89c5c62f2 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:24.940479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f83a19-39d3-4cea-98cb-017a9bad4acf None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:24.940755 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f83a19-39d3-4cea-98cb-017a9bad4acf None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:24.941033 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f83a19-39d3-4cea-98cb-017a9bad4acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:24.961734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-05f83a19-39d3-4cea-98cb-017a9bad4acf None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:24.962989 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-05f83a19-39d3-4cea-98cb-017a9bad4acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:24.986295 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cc123-0487-4de0-b1de-b81ea7540c67 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:24.991303 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad1a19-4686-4de2-b3ac-dcc15bdf1c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:24.991940 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad1a19-4686-4de2-b3ac-dcc15bdf1c84 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:24.992230 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad1a19-4686-4de2-b3ac-dcc15bdf1c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:24.992508 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad1a19-4686-4de2-b3ac-dcc15bdf1c84 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:24.993943 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad1a19-4686-4de2-b3ac-dcc15bdf1c84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:24.996035 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad1a19-4686-4de2-b3ac-dcc15bdf1c84 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:25.033029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1012/1996] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:25.040941 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e94130-f6e5-4076-add3-d07a4fe21e76 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:25.041229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e94130-f6e5-4076-add3-d07a4fe21e76 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:25.041508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e94130-f6e5-4076-add3-d07a4fe21e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:25.061450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-94e94130-f6e5-4076-add3-d07a4fe21e76 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:25.062684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-94e94130-f6e5-4076-add3-d07a4fe21e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:25.071311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1013/1997] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:25.075533 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e94130-f6e5-4076-add3-d07a4fe21e76 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:25.085330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e94130-f6e5-4076-add3-d07a4fe21e76 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:25.093270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f40998-cd3c-4b02-9cad-413a10af836f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:25.093270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f40998-cd3c-4b02-9cad-413a10af836f admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:25.093270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f40998-cd3c-4b02-9cad-413a10af836f admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:25.093270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f40998-cd3c-4b02-9cad-413a10af836f admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:25.093684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f40998-cd3c-4b02-9cad-413a10af836f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:36:25.093684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f40998-cd3c-4b02-9cad-413a10af836f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:25.095190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f40998-cd3c-4b02-9cad-413a10af836f admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:25.109976 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 985/1998] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:24 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:25.114277 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad1a19-4686-4de2-b3ac-dcc15bdf1c84 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:25.114421 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1014/1999] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:36:25.116262 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f40998-cd3c-4b02-9cad-413a10af836f admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:25.116433 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-50ad1a19-4686-4de2-b3ac-dcc15bdf1c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:25.128564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad1a19-4686-4de2-b3ac-dcc15bdf1c84 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:25.129369 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f40998-cd3c-4b02-9cad-413a10af836f admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:25.135371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c5054-4a09-4eb3-b7ab-b34b33b55221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:25.136118 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c5054-4a09-4eb3-b7ab-b34b33b55221 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:25.136235 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a529d4b-9a7b-4b0a-b78c-ca77520f8cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:25.136595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c5054-4a09-4eb3-b7ab-b34b33b55221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:25.136805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a529d4b-9a7b-4b0a-b78c-ca77520f8cfc admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:25.137086 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c5054-4a09-4eb3-b7ab-b34b33b55221 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:25.137206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a529d4b-9a7b-4b0a-b78c-ca77520f8cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:25.137361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a529d4b-9a7b-4b0a-b78c-ca77520f8cfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:25.138007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c5054-4a09-4eb3-b7ab-b34b33b55221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:25.138123 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a529d4b-9a7b-4b0a-b78c-ca77520f8cfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:25.139821 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c5054-4a09-4eb3-b7ab-b34b33b55221 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:25.139937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a529d4b-9a7b-4b0a-b78c-ca77520f8cfc admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:25.152710 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 986/2000] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:25.157126 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c5054-4a09-4eb3-b7ab-b34b33b55221 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:25.167178 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c5054-4a09-4eb3-b7ab-b34b33b55221 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:25.171855 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a80b2-9929-4eb2-84fc-c82d38c9d093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:25.172561 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a80b2-9929-4eb2-84fc-c82d38c9d093 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:25.172846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a80b2-9929-4eb2-84fc-c82d38c9d093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:25.173125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a80b2-9929-4eb2-84fc-c82d38c9d093 admin admin] PATH_INFO: `/v3/projects/a0dabe71268b4ecd963b98aedb1a77d7/users/ad69761008f143c9b7e34d47f59d9bc3/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:25.188424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a80b2-9929-4eb2-84fc-c82d38c9d093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0dabe71268b4ecd963b98aedb1a77d7, user_id=ad69761008f143c9b7e34d47f59d9bc3, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:25.191184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a80b2-9929-4eb2-84fc-c82d38c9d093 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:25.314939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1015/2001] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:25 2022] POST /identity/v3/projects => generated 393 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:25.315878 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 987/2002] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:25 2022] PUT /identity/v3/projects/a0dabe71268b4ecd963b98aedb1a77d7/users/ad69761008f143c9b7e34d47f59d9bc3/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:25.320197 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a80b2-9929-4eb2-84fc-c82d38c9d093 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:25.320336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a529d4b-9a7b-4b0a-b78c-ca77520f8cfc admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:25.382679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:25.384381 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a529d4b-9a7b-4b0a-b78c-ca77520f8cfc admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:25.387906 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a80b2-9929-4eb2-84fc-c82d38c9d093 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:25.389299 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806b76d-7508-4611-a887-50ebb6b6d8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:25.389912 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806b76d-7508-4611-a887-50ebb6b6d8b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:25.390199 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806b76d-7508-4611-a887-50ebb6b6d8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:25.391703 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806b76d-7508-4611-a887-50ebb6b6d8b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:25.391703 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9806b76d-7508-4611-a887-50ebb6b6d8b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:25.393535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c0361-6cba-4469-94b1-ed8cd4ed9e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:25.393655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9806b76d-7508-4611-a887-50ebb6b6d8b7 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:25.394145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c0361-6cba-4469-94b1-ed8cd4ed9e15 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:25.394433 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c0361-6cba-4469-94b1-ed8cd4ed9e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:25.394710 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c0361-6cba-4469-94b1-ed8cd4ed9e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:25.395387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c0361-6cba-4469-94b1-ed8cd4ed9e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:25.396985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c0361-6cba-4469-94b1-ed8cd4ed9e15 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:25.407394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 988/2003] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:25.412034 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c0361-6cba-4469-94b1-ed8cd4ed9e15 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:25.423220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c0361-6cba-4469-94b1-ed8cd4ed9e15 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:25.432332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0308149-4023-4fef-9b5f-77bcf975dba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:25.433149 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0308149-4023-4fef-9b5f-77bcf975dba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:25.433501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0308149-4023-4fef-9b5f-77bcf975dba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:25.433844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0308149-4023-4fef-9b5f-77bcf975dba7 admin admin] PATH_INFO: `/v3/projects/a0dabe71268b4ecd963b98aedb1a77d7/users/ad69761008f143c9b7e34d47f59d9bc3/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:25.437221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0308149-4023-4fef-9b5f-77bcf975dba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0dabe71268b4ecd963b98aedb1a77d7, user_id=ad69761008f143c9b7e34d47f59d9bc3, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:25.442191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0308149-4023-4fef-9b5f-77bcf975dba7 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:25.463056 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 989/2004] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:25 2022] PUT /identity/v3/projects/a0dabe71268b4ecd963b98aedb1a77d7/users/ad69761008f143c9b7e34d47f59d9bc3/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:25.464591 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afeaf9-aa43-47c6-aac5-ff73a8500eae None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:25.464961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afeaf9-aa43-47c6-aac5-ff73a8500eae None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:25.465567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afeaf9-aa43-47c6-aac5-ff73a8500eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:25.502689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-76afeaf9-aa43-47c6-aac5-ff73a8500eae None None] MFA Rules not processed for user `ad69761008f143c9b7e34d47f59d9bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:25.503979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-76afeaf9-aa43-47c6-aac5-ff73a8500eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:25.537815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1016/2005] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:25 2022] POST /identity/v3/users => generated 297 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:25.544696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806b76d-7508-4611-a887-50ebb6b6d8b7 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:25.556039 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806b76d-7508-4611-a887-50ebb6b6d8b7 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:25.598127 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 990/2006] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:25.608995 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afeaf9-aa43-47c6-aac5-ff73a8500eae None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:25.611166 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-76afeaf9-aa43-47c6-aac5-ff73a8500eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:25.655692 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe65cb-3fec-4966-a5b0-2070bfdd9e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:25.657158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe65cb-3fec-4966-a5b0-2070bfdd9e22 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:25.657501 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe65cb-3fec-4966-a5b0-2070bfdd9e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:25.657797 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe65cb-3fec-4966-a5b0-2070bfdd9e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:25.658536 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe65cb-3fec-4966-a5b0-2070bfdd9e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:25.660275 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe65cb-3fec-4966-a5b0-2070bfdd9e22 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:25.670715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1017/2007] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:25.675188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe65cb-3fec-4966-a5b0-2070bfdd9e22 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:25.679432 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afeaf9-aa43-47c6-aac5-ff73a8500eae None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:25.683228 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670435b8-dcf5-4d56-97c6-8beffbe7dfaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:25.683850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670435b8-dcf5-4d56-97c6-8beffbe7dfaf service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:25.684134 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670435b8-dcf5-4d56-97c6-8beffbe7dfaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:25.684415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670435b8-dcf5-4d56-97c6-8beffbe7dfaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:25.685803 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe65cb-3fec-4966-a5b0-2070bfdd9e22 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:25.690639 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38607330-ac41-4487-b5c2-b16d4982d0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:25.691319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38607330-ac41-4487-b5c2-b16d4982d0b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:25.691603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38607330-ac41-4487-b5c2-b16d4982d0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:25.691878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38607330-ac41-4487-b5c2-b16d4982d0b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:25.692677 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38607330-ac41-4487-b5c2-b16d4982d0b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:25.694257 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38607330-ac41-4487-b5c2-b16d4982d0b0 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:25.694521 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670435b8-dcf5-4d56-97c6-8beffbe7dfaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:25.696006 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670435b8-dcf5-4d56-97c6-8beffbe7dfaf service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:25.708817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1018/2008] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:25.709683 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38607330-ac41-4487-b5c2-b16d4982d0b0 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:25.720143 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 991/2009] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:36:25.720872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38607330-ac41-4487-b5c2-b16d4982d0b0 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:25.729664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:25.732282 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44ee0e-30e6-4f39-90c2-ae8ba100d46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:25.732996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44ee0e-30e6-4f39-90c2-ae8ba100d46b admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:25.733298 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44ee0e-30e6-4f39-90c2-ae8ba100d46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:25.733687 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44ee0e-30e6-4f39-90c2-ae8ba100d46b admin admin] PATH_INFO: `/v3/projects/44b606ca920944cc9b6773fc5f0678d5/users/5ca108cd9b064d729b047d4d2417e2be/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:25.748625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad44ee0e-30e6-4f39-90c2-ae8ba100d46b admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b606ca920944cc9b6773fc5f0678d5, user_id=5ca108cd9b064d729b047d4d2417e2be, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:25.752002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad44ee0e-30e6-4f39-90c2-ae8ba100d46b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:25.835978 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1019/2010] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:25 2022] PUT /identity/v3/projects/44b606ca920944cc9b6773fc5f0678d5/users/5ca108cd9b064d729b047d4d2417e2be/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:25.846311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57e031-51bf-4ffd-80e1-d320c64be00c None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:25.846604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57e031-51bf-4ffd-80e1-d320c64be00c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:25.846921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57e031-51bf-4ffd-80e1-d320c64be00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:25.879272 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-4b57e031-51bf-4ffd-80e1-d320c64be00c None None] MFA Rules not processed for user `5ca108cd9b064d729b047d4d2417e2be`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:25.880372 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-4b57e031-51bf-4ffd-80e1-d320c64be00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:25.960184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 992/2011] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:25.968109 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44ee0e-30e6-4f39-90c2-ae8ba100d46b admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:26.042609 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44ee0e-30e6-4f39-90c2-ae8ba100d46b admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:26.047840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb5491-5ff5-48b6-8186-a344096181aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:26.048498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb5491-5ff5-48b6-8186-a344096181aa admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:26.048787 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb5491-5ff5-48b6-8186-a344096181aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:26.049249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb5491-5ff5-48b6-8186-a344096181aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:26.051205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb5491-5ff5-48b6-8186-a344096181aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:26.051970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb5491-5ff5-48b6-8186-a344096181aa admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:26.272806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1020/2012] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:25 2022] POST /identity/v3/projects => generated 391 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:26.288169 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57e031-51bf-4ffd-80e1-d320c64be00c None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:26.300203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57e031-51bf-4ffd-80e1-d320c64be00c None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:26.367866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51e32c-3759-4a7b-93d4-7a7519e9aefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:26.368508 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51e32c-3759-4a7b-93d4-7a7519e9aefc admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:26.368796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51e32c-3759-4a7b-93d4-7a7519e9aefc admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:26.369190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51e32c-3759-4a7b-93d4-7a7519e9aefc admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:26.370193 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de51e32c-3759-4a7b-93d4-7a7519e9aefc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:26.373383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de51e32c-3759-4a7b-93d4-7a7519e9aefc admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:26.493817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 993/2013] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:26 2022] POST /identity/v3/users => generated 302 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:26.495133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb5491-5ff5-48b6-8186-a344096181aa admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:26.506004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb5491-5ff5-48b6-8186-a344096181aa admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:26.511075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2023939-64c0-4921-ae34-ef1d716778c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:26.512310 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2023939-64c0-4921-ae34-ef1d716778c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:26.512595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2023939-64c0-4921-ae34-ef1d716778c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:26.512874 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2023939-64c0-4921-ae34-ef1d716778c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:26.519247 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2023939-64c0-4921-ae34-ef1d716778c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:26.522447 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2023939-64c0-4921-ae34-ef1d716778c4 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:26.540566 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1021/2014] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:26.542221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51e32c-3759-4a7b-93d4-7a7519e9aefc admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:26.557530 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51e32c-3759-4a7b-93d4-7a7519e9aefc admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:26.558545 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5acf8d-f0cc-4a77-bb3d-46564d168d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:26.559637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5acf8d-f0cc-4a77-bb3d-46564d168d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:26.559944 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5acf8d-f0cc-4a77-bb3d-46564d168d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:26.560224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5acf8d-f0cc-4a77-bb3d-46564d168d43 admin admin] PATH_INFO: `/v3/projects/3e94be89fdd144f0b1ea241471e7cbf8/users/4d1c2d3fc2e9435098ff52d45e9522d1/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:26.577382 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5acf8d-f0cc-4a77-bb3d-46564d168d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e94be89fdd144f0b1ea241471e7cbf8, user_id=4d1c2d3fc2e9435098ff52d45e9522d1, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:26.591000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5acf8d-f0cc-4a77-bb3d-46564d168d43 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:26.633854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:26.683502 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 994/2015] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:26 2022] PUT /identity/v3/projects/3e94be89fdd144f0b1ea241471e7cbf8/users/4d1c2d3fc2e9435098ff52d45e9522d1/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:26.687195 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2023939-64c0-4921-ae34-ef1d716778c4 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:26.700004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2023939-64c0-4921-ae34-ef1d716778c4 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:26.763572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1439428-fb3a-4b3a-9a76-d3a60e80da34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:26.764491 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1439428-fb3a-4b3a-9a76-d3a60e80da34 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:26.764924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1439428-fb3a-4b3a-9a76-d3a60e80da34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:26.765464 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1439428-fb3a-4b3a-9a76-d3a60e80da34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:26.767200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1439428-fb3a-4b3a-9a76-d3a60e80da34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:26.769499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1439428-fb3a-4b3a-9a76-d3a60e80da34 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:26.790333 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1022/2016] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:26.794714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5acf8d-f0cc-4a77-bb3d-46564d168d43 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:26.807087 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5acf8d-f0cc-4a77-bb3d-46564d168d43 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:26.811968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60e00c-2fd9-4794-8c47-cc91d1b5ed0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:26.812651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60e00c-2fd9-4794-8c47-cc91d1b5ed0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:26.813464 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60e00c-2fd9-4794-8c47-cc91d1b5ed0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:26.813739 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60e00c-2fd9-4794-8c47-cc91d1b5ed0a admin admin] PATH_INFO: `/v3/projects/3e94be89fdd144f0b1ea241471e7cbf8/users/4d1c2d3fc2e9435098ff52d45e9522d1/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:26.815738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60e00c-2fd9-4794-8c47-cc91d1b5ed0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e94be89fdd144f0b1ea241471e7cbf8, user_id=4d1c2d3fc2e9435098ff52d45e9522d1, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:26.818414 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60e00c-2fd9-4794-8c47-cc91d1b5ed0a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:26.832154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 995/2017] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:26 2022] PUT /identity/v3/projects/3e94be89fdd144f0b1ea241471e7cbf8/users/4d1c2d3fc2e9435098ff52d45e9522d1/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:26.844568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baaefef-029e-4ea4-8047-5d1309a8da0a None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:26.844857 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baaefef-029e-4ea4-8047-5d1309a8da0a None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:26.845140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baaefef-029e-4ea4-8047-5d1309a8da0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:26.855478 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:26.884466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-5baaefef-029e-4ea4-8047-5d1309a8da0a None None] MFA Rules not processed for user `4d1c2d3fc2e9435098ff52d45e9522d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:26.886204 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5baaefef-029e-4ea4-8047-5d1309a8da0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:26.972101 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1023/2018] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:26 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:26.984010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60e00c-2fd9-4794-8c47-cc91d1b5ed0a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:27.045999 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60e00c-2fd9-4794-8c47-cc91d1b5ed0a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:27.050574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6f534-7a76-4a15-a540-4a7397b54b0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:27.051228 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6f534-7a76-4a15-a540-4a7397b54b0c service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:27.051511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6f534-7a76-4a15-a540-4a7397b54b0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:27.051785 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6f534-7a76-4a15-a540-4a7397b54b0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:27.061532 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6f534-7a76-4a15-a540-4a7397b54b0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:27.063549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6f534-7a76-4a15-a540-4a7397b54b0c service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:27.073320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 996/2019] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:26 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:27.117066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baaefef-029e-4ea4-8047-5d1309a8da0a None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:27.133120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6f534-7a76-4a15-a540-4a7397b54b0c service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:27.148990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6f534-7a76-4a15-a540-4a7397b54b0c service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:27.151273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7fddb-4b96-4413-9aea-0f3abb9d4b8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:27.152283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7fddb-4b96-4413-9aea-0f3abb9d4b8e service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:27.152686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7fddb-4b96-4413-9aea-0f3abb9d4b8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:27.153124 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7fddb-4b96-4413-9aea-0f3abb9d4b8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:27.189791 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baaefef-029e-4ea4-8047-5d1309a8da0a None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:27.195212 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f5d55-394a-4642-b9da-cafae81d921f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:27.195971 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f5d55-394a-4642-b9da-cafae81d921f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:27.196321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f5d55-394a-4642-b9da-cafae81d921f admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:27.196601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f5d55-394a-4642-b9da-cafae81d921f admin admin] PATH_INFO: `/v3/users/ba2dd9647d94417d800daa14ee56fdef` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:27.197763 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f5d55-394a-4642-b9da-cafae81d921f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba2dd9647d94417d800daa14ee56fdef)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:27.199761 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f5d55-394a-4642-b9da-cafae81d921f admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:27.237797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7fddb-4b96-4413-9aea-0f3abb9d4b8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:27.239296 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7fddb-4b96-4413-9aea-0f3abb9d4b8e service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:27.255319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 997/2020] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:27 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:36:27.255319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7fddb-4b96-4413-9aea-0f3abb9d4b8e service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:27.255319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ead7fddb-4b96-4413-9aea-0f3abb9d4b8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:27.265619 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7fddb-4b96-4413-9aea-0f3abb9d4b8e service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:27.270512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60315d59-33ae-4684-aeaa-b428c0735aae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:27.270824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60315d59-33ae-4684-aeaa-b428c0735aae service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:27.271145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60315d59-33ae-4684-aeaa-b428c0735aae service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:27.271422 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60315d59-33ae-4684-aeaa-b428c0735aae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:27.273984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-60315d59-33ae-4684-aeaa-b428c0735aae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:27.349760 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60315d59-33ae-4684-aeaa-b428c0735aae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:27.351385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60315d59-33ae-4684-aeaa-b428c0735aae service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:27.362236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 998/2021] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:27.412060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:27.414458 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-505f5d55-394a-4642-b9da-cafae81d921f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba2dd9647d94417d800daa14ee56fdef'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:27.519792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-505f5d55-394a-4642-b9da-cafae81d921f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba2dd9647d94417d800daa14ee56fdef'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:27.533918 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-505f5d55-394a-4642-b9da-cafae81d921f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba2dd9647d94417d800daa14ee56fdef'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:27.534989 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-505f5d55-394a-4642-b9da-cafae81d921f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba2dd9647d94417d800daa14ee56fdef'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:27.542051 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-505f5d55-394a-4642-b9da-cafae81d921f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba2dd9647d94417d800daa14ee56fdef'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:27.548826 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1024/2022] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:27 2022] DELETE /identity/v3/users/ba2dd9647d94417d800daa14ee56fdef => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:27.553652 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fbbf9-cd5b-4ebf-8d7c-e5bc84ec38ce None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:27.553937 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fbbf9-cd5b-4ebf-8d7c-e5bc84ec38ce None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:27.554294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fbbf9-cd5b-4ebf-8d7c-e5bc84ec38ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:27.574763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-3a1fbbf9-cd5b-4ebf-8d7c-e5bc84ec38ce None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:27.576084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-3a1fbbf9-cd5b-4ebf-8d7c-e5bc84ec38ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:27.656086 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 999/2023] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:27.670149 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f5d55-394a-4642-b9da-cafae81d921f admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:27.670149 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-505f5d55-394a-4642-b9da-cafae81d921f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:27.740595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f5d55-394a-4642-b9da-cafae81d921f admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:27.745572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a52887-4991-45e0-8882-ed771aecb904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:27.746034 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a52887-4991-45e0-8882-ed771aecb904 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:27.746388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a52887-4991-45e0-8882-ed771aecb904 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:27.746730 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a52887-4991-45e0-8882-ed771aecb904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:27.758281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a52887-4991-45e0-8882-ed771aecb904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:27.760057 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a52887-4991-45e0-8882-ed771aecb904 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:27.781098 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1025/2024] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:28.407020 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=75,keystone:DELETE=23,keystone:INSERT=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:28.697707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1fbbf9-cd5b-4ebf-8d7c-e5bc84ec38ce None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:28.699221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-3a1fbbf9-cd5b-4ebf-8d7c-e5bc84ec38ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:28.713219 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1fbbf9-cd5b-4ebf-8d7c-e5bc84ec38ce None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:28.718355 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:28.719916 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:28.720268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:28.720725 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a admin admin] PATH_INFO: `/v3/projects/ada19ac8bad54797b8b43350164d3eee` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:28.721858 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ada19ac8bad54797b8b43350164d3eee)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:28.724004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:28.881371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ada19ac8bad54797b8b43350164d3eee', 'request_id': 'req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:28.886947 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ada19ac8bad54797b8b43350164d3eee', 'request_id': 'req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:29.007227 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ada19ac8bad54797b8b43350164d3eee', 'request_id': 'req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:29.019894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a admin admin] The token cache is being invalidate because project ada19ac8bad54797b8b43350164d3eee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:29.020181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:29.020806 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:29.022377 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1000/2025] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:28 2022] DELETE /identity/v3/projects/ada19ac8bad54797b8b43350164d3eee => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:29.026418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a52887-4991-45e0-8882-ed771aecb904 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:29.036118 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a52887-4991-45e0-8882-ed771aecb904 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:29.100071 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb7f45-30c4-4bb2-824e-628e5c803b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:29.100709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb7f45-30c4-4bb2-824e-628e5c803b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:29.100992 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb7f45-30c4-4bb2-824e-628e5c803b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:29.101269 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb7f45-30c4-4bb2-824e-628e5c803b31 admin admin] PATH_INFO: `/v3/users/ad69761008f143c9b7e34d47f59d9bc3` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:29.102421 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb7f45-30c4-4bb2-824e-628e5c803b31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad69761008f143c9b7e34d47f59d9bc3)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:29.104403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb7f45-30c4-4bb2-824e-628e5c803b31 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:29.315730 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:29.359330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-58fb7f45-30c4-4bb2-824e-628e5c803b31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad69761008f143c9b7e34d47f59d9bc3'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:29.474344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-58fb7f45-30c4-4bb2-824e-628e5c803b31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad69761008f143c9b7e34d47f59d9bc3'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:29.501730 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-58fb7f45-30c4-4bb2-824e-628e5c803b31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad69761008f143c9b7e34d47f59d9bc3'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:29.504129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-58fb7f45-30c4-4bb2-824e-628e5c803b31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad69761008f143c9b7e34d47f59d9bc3'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:29.521590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-58fb7f45-30c4-4bb2-824e-628e5c803b31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad69761008f143c9b7e34d47f59d9bc3'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:29.541277 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1026/2026] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:29 2022] DELETE /identity/v3/users/ad69761008f143c9b7e34d47f59d9bc3 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:30.256795 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:30.317169 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:30.318677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:30.332245 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ce327-21f5-4a8c-8e5f-3228c3aa964a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:30.392320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9cb01-f8fd-4d16-9141-d30e40bd0fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:30.392959 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9cb01-f8fd-4d16-9141-d30e40bd0fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:30.393240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9cb01-f8fd-4d16-9141-d30e40bd0fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:30.393517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9cb01-f8fd-4d16-9141-d30e40bd0fb3 admin admin] PATH_INFO: `/v3/projects/a0dabe71268b4ecd963b98aedb1a77d7` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:30.394627 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9cb01-f8fd-4d16-9141-d30e40bd0fb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0dabe71268b4ecd963b98aedb1a77d7)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:30.396582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9cb01-f8fd-4d16-9141-d30e40bd0fb3 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:30.554274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a5a9cb01-f8fd-4d16-9141-d30e40bd0fb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0dabe71268b4ecd963b98aedb1a77d7', 'request_id': 'req-a5a9cb01-f8fd-4d16-9141-d30e40bd0fb3'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:30.560780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a5a9cb01-f8fd-4d16-9141-d30e40bd0fb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0dabe71268b4ecd963b98aedb1a77d7', 'request_id': 'req-a5a9cb01-f8fd-4d16-9141-d30e40bd0fb3'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:30.704976 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a5a9cb01-f8fd-4d16-9141-d30e40bd0fb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0dabe71268b4ecd963b98aedb1a77d7', 'request_id': 'req-a5a9cb01-f8fd-4d16-9141-d30e40bd0fb3'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:30.717794 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a5a9cb01-f8fd-4d16-9141-d30e40bd0fb3 admin admin] The token cache is being invalidate because project a0dabe71268b4ecd963b98aedb1a77d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:30.718084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a5a9cb01-f8fd-4d16-9141-d30e40bd0fb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:30.719105 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a5a9cb01-f8fd-4d16-9141-d30e40bd0fb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:30.724957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1001/2027] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:30 2022] DELETE /identity/v3/projects/a0dabe71268b4ecd963b98aedb1a77d7 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:30.725998 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d2d8f-d875-447a-9691-be8bf332a29c None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:30.726459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d2d8f-d875-447a-9691-be8bf332a29c None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:30.726572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d2d8f-d875-447a-9691-be8bf332a29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:30.752108 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-f45d2d8f-d875-447a-9691-be8bf332a29c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:30.753543 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f45d2d8f-d875-447a-9691-be8bf332a29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:30.858145 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1027/2028] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:30.868005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83487a3-59b8-4b7e-bb35-89fc75c3a6aa None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:30.868298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83487a3-59b8-4b7e-bb35-89fc75c3a6aa None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:30.868579 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83487a3-59b8-4b7e-bb35-89fc75c3a6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:30.902739 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-e83487a3-59b8-4b7e-bb35-89fc75c3a6aa None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:30.902739 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e83487a3-59b8-4b7e-bb35-89fc75c3a6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:30.909667 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1002/2029] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:30.913634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d2d8f-d875-447a-9691-be8bf332a29c None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:30.929881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d2d8f-d875-447a-9691-be8bf332a29c None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:30.943164 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2104f-d149-400c-90e1-61d051724d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:30.943818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2104f-d149-400c-90e1-61d051724d3d admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:30.944075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2104f-d149-400c-90e1-61d051724d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:30.944334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2104f-d149-400c-90e1-61d051724d3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:30.945243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2104f-d149-400c-90e1-61d051724d3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:36:30.945742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2104f-d149-400c-90e1-61d051724d3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:30.947783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2104f-d149-400c-90e1-61d051724d3d admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:30.964843 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1028/2030] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:36:30.970419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83487a3-59b8-4b7e-bb35-89fc75c3a6aa None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:30.994448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83487a3-59b8-4b7e-bb35-89fc75c3a6aa None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:31.021117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e2daa-f044-4b47-ace3-7c98cf100d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:31.021984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e2daa-f044-4b47-ace3-7c98cf100d9d admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:31.022098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e2daa-f044-4b47-ace3-7c98cf100d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:31.025289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e2daa-f044-4b47-ace3-7c98cf100d9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:31.025955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e2daa-f044-4b47-ace3-7c98cf100d9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:31.027989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e2daa-f044-4b47-ace3-7c98cf100d9d admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:31.151733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1003/2031] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:30 2022] POST /identity/v3/projects => generated 407 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:31.156192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2104f-d149-400c-90e1-61d051724d3d admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:31.167623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2104f-d149-400c-90e1-61d051724d3d admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:31.236618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d05837-828c-4b0f-b239-0fc8b57160d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:31.237368 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d05837-828c-4b0f-b239-0fc8b57160d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:31.237650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d05837-828c-4b0f-b239-0fc8b57160d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:31.237922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d05837-828c-4b0f-b239-0fc8b57160d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:31.239176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d05837-828c-4b0f-b239-0fc8b57160d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:31.241706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d05837-828c-4b0f-b239-0fc8b57160d5 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:31.302744 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:31.361528 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1029/2032] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:31 2022] POST /identity/v3/users => generated 304 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:31.365979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e2daa-f044-4b47-ace3-7c98cf100d9d admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:31.380074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e2daa-f044-4b47-ace3-7c98cf100d9d admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:31.385874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bcc353-3b04-4770-ac93-62e7ff38a63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:31.387106 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcc353-3b04-4770-ac93-62e7ff38a63c admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:31.387606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcc353-3b04-4770-ac93-62e7ff38a63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:31.388108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcc353-3b04-4770-ac93-62e7ff38a63c admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:31.388969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bcc353-3b04-4770-ac93-62e7ff38a63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:31.391491 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bcc353-3b04-4770-ac93-62e7ff38a63c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:31.409265 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d05837-828c-4b0f-b239-0fc8b57160d5 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:31.409724 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1004/2033] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:31.419877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d05837-828c-4b0f-b239-0fc8b57160d5 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:31.426192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4b208-6440-4ce8-bbd6-e028d0a98e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:31.432354 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4b208-6440-4ce8-bbd6-e028d0a98e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:31.432354 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4b208-6440-4ce8-bbd6-e028d0a98e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:31.432354 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4b208-6440-4ce8-bbd6-e028d0a98e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:31.432354 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4b208-6440-4ce8-bbd6-e028d0a98e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:31.432354 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4b208-6440-4ce8-bbd6-e028d0a98e4f admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:31.445907 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1030/2034] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:31.447159 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bcc353-3b04-4770-ac93-62e7ff38a63c admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:31.457988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bcc353-3b04-4770-ac93-62e7ff38a63c admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:31.466847 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c29dfc-0f15-46ab-a116-671d5ca6d182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:31.466847 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c29dfc-0f15-46ab-a116-671d5ca6d182 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:31.466847 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c29dfc-0f15-46ab-a116-671d5ca6d182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:31.466847 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c29dfc-0f15-46ab-a116-671d5ca6d182 admin admin] PATH_INFO: `/v3/projects/eea97abb092249868b4aaef4d8f0c4c3/users/49192fcffa9e41b5971cdb1b337f8c9a/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:31.480032 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c29dfc-0f15-46ab-a116-671d5ca6d182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea97abb092249868b4aaef4d8f0c4c3, user_id=49192fcffa9e41b5971cdb1b337f8c9a, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:31.482774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c29dfc-0f15-46ab-a116-671d5ca6d182 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:31.618738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1005/2035] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:31 2022] PUT /identity/v3/projects/eea97abb092249868b4aaef4d8f0c4c3/users/49192fcffa9e41b5971cdb1b337f8c9a/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:31.629254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecf8f8-db40-4509-95ae-ae26cb75752e None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:31.629615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecf8f8-db40-4509-95ae-ae26cb75752e None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:31.630194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecf8f8-db40-4509-95ae-ae26cb75752e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:31.662403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-74ecf8f8-db40-4509-95ae-ae26cb75752e None None] MFA Rules not processed for user `49192fcffa9e41b5971cdb1b337f8c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:31.663881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-74ecf8f8-db40-4509-95ae-ae26cb75752e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:31.755104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1031/2036] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:31 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:31.759599 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c29dfc-0f15-46ab-a116-671d5ca6d182 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:31.844752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c29dfc-0f15-46ab-a116-671d5ca6d182 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:31.851164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ac5f8-af1c-4be0-9c03-c145bb67819a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:31.851783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ac5f8-af1c-4be0-9c03-c145bb67819a admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:31.852090 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ac5f8-af1c-4be0-9c03-c145bb67819a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:31.852363 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ac5f8-af1c-4be0-9c03-c145bb67819a admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:31.852990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ac5f8-af1c-4be0-9c03-c145bb67819a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:31.855390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ac5f8-af1c-4be0-9c03-c145bb67819a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:31.981956 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1006/2037] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:31 2022] POST /identity/v3/projects => generated 407 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:31.985449 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecf8f8-db40-4509-95ae-ae26cb75752e None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:32.002431 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecf8f8-db40-4509-95ae-ae26cb75752e None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:32.070468 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b2a31-02ef-40ab-8969-1338e3cf2465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:32.070468 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b2a31-02ef-40ab-8969-1338e3cf2465 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:32.071454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b2a31-02ef-40ab-8969-1338e3cf2465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:32.071776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ac5f8-af1c-4be0-9c03-c145bb67819a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:32.072373 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b2a31-02ef-40ab-8969-1338e3cf2465 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:32.075709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b2a31-02ef-40ab-8969-1338e3cf2465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:32.079983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-071ac5f8-af1c-4be0-9c03-c145bb67819a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:32.080373 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b2a31-02ef-40ab-8969-1338e3cf2465 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:32.096842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ac5f8-af1c-4be0-9c03-c145bb67819a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:32.102077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb745704-8b2f-45fc-867c-5935789e2550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:32.102766 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb745704-8b2f-45fc-867c-5935789e2550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:32.103016 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb745704-8b2f-45fc-867c-5935789e2550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:32.103294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb745704-8b2f-45fc-867c-5935789e2550 admin admin] PATH_INFO: `/v3/users/d31427f2a67e49d0b4d3e1b3cee8c48b` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:32.104420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb745704-8b2f-45fc-867c-5935789e2550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d31427f2a67e49d0b4d3e1b3cee8c48b)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:32.106379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb745704-8b2f-45fc-867c-5935789e2550 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:32.256825 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1032/2038] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:31 2022] POST /identity/v3/users => generated 310 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:32.257939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b2a31-02ef-40ab-8969-1338e3cf2465 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:32.268878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b2a31-02ef-40ab-8969-1338e3cf2465 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:32.273839 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5cc7e-c7a8-4452-8b90-beae313b1f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:32.274459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5cc7e-c7a8-4452-8b90-beae313b1f77 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:32.274739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5cc7e-c7a8-4452-8b90-beae313b1f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:32.275053 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5cc7e-c7a8-4452-8b90-beae313b1f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:32.275701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5cc7e-c7a8-4452-8b90-beae313b1f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:32.277298 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5cc7e-c7a8-4452-8b90-beae313b1f77 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:32.288074 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1033/2039] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:32.292173 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5cc7e-c7a8-4452-8b90-beae313b1f77 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:32.302130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5cc7e-c7a8-4452-8b90-beae313b1f77 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:32.306886 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70b712-ae36-4c2c-a8be-caec685e5e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:32.307705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70b712-ae36-4c2c-a8be-caec685e5e2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:32.307999 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70b712-ae36-4c2c-a8be-caec685e5e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:32.308278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70b712-ae36-4c2c-a8be-caec685e5e2d admin admin] PATH_INFO: `/v3/projects/57bf2f52c9214b9dbd69af9981905a95/users/5baea20e78174f92a0c6bfb3ab14741f/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:32.324034 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70b712-ae36-4c2c-a8be-caec685e5e2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=57bf2f52c9214b9dbd69af9981905a95, user_id=5baea20e78174f92a0c6bfb3ab14741f, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:32.326702 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70b712-ae36-4c2c-a8be-caec685e5e2d admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:32.450952 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1034/2040] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:32 2022] PUT /identity/v3/projects/57bf2f52c9214b9dbd69af9981905a95/users/5baea20e78174f92a0c6bfb3ab14741f/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:32.450952 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70b712-ae36-4c2c-a8be-caec685e5e2d admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:32.467328 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-bb745704-8b2f-45fc-867c-5935789e2550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd31427f2a67e49d0b4d3e1b3cee8c48b'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:32.528633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70b712-ae36-4c2c-a8be-caec685e5e2d admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:32.535094 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225a5e4-7c23-49d4-804e-52e4f8f9d50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:32.535806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225a5e4-7c23-49d4-804e-52e4f8f9d50a admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:32.536100 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225a5e4-7c23-49d4-804e-52e4f8f9d50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:32.536373 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225a5e4-7c23-49d4-804e-52e4f8f9d50a admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:32.537026 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225a5e4-7c23-49d4-804e-52e4f8f9d50a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:32.538625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225a5e4-7c23-49d4-804e-52e4f8f9d50a admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:32.549484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1035/2041] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:32.553769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225a5e4-7c23-49d4-804e-52e4f8f9d50a admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:32.564261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225a5e4-7c23-49d4-804e-52e4f8f9d50a admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:32.568938 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664972a8-7e80-4e10-ba9b-37c13cf3e3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:32.569588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664972a8-7e80-4e10-ba9b-37c13cf3e3f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:32.569871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664972a8-7e80-4e10-ba9b-37c13cf3e3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:32.570148 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664972a8-7e80-4e10-ba9b-37c13cf3e3f7 admin admin] PATH_INFO: `/v3/projects/57bf2f52c9214b9dbd69af9981905a95/users/5baea20e78174f92a0c6bfb3ab14741f/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:32.572965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664972a8-7e80-4e10-ba9b-37c13cf3e3f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57bf2f52c9214b9dbd69af9981905a95, user_id=5baea20e78174f92a0c6bfb3ab14741f, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:32.575652 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664972a8-7e80-4e10-ba9b-37c13cf3e3f7 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:32.580339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-bb745704-8b2f-45fc-867c-5935789e2550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd31427f2a67e49d0b4d3e1b3cee8c48b'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:32.581309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-bb745704-8b2f-45fc-867c-5935789e2550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd31427f2a67e49d0b4d3e1b3cee8c48b'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:32.595177 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1036/2042] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:32 2022] PUT /identity/v3/projects/57bf2f52c9214b9dbd69af9981905a95/users/5baea20e78174f92a0c6bfb3ab14741f/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:32.604985 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f46310-b6c9-4328-b0b4-374e6908dfec None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:32.605377 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f46310-b6c9-4328-b0b4-374e6908dfec None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:32.605655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f46310-b6c9-4328-b0b4-374e6908dfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:32.613007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:32.618154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-bb745704-8b2f-45fc-867c-5935789e2550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd31427f2a67e49d0b4d3e1b3cee8c48b'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:32.628005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-bb745704-8b2f-45fc-867c-5935789e2550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd31427f2a67e49d0b4d3e1b3cee8c48b'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:32.636271 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1007/2043] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:32 2022] DELETE /identity/v3/users/d31427f2a67e49d0b4d3e1b3cee8c48b => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:32.655778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-32f46310-b6c9-4328-b0b4-374e6908dfec None None] MFA Rules not processed for user `5baea20e78174f92a0c6bfb3ab14741f`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:32.657061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-32f46310-b6c9-4328-b0b4-374e6908dfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:32.732407 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1037/2044] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:32 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:32.743490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb745704-8b2f-45fc-867c-5935789e2550 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:32.745064 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-bb745704-8b2f-45fc-867c-5935789e2550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:32.805119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb745704-8b2f-45fc-867c-5935789e2550 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:32.808918 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffd25b-6c41-40c9-8e9d-da9a2a5646c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:32.809537 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffd25b-6c41-40c9-8e9d-da9a2a5646c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:32.809946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffd25b-6c41-40c9-8e9d-da9a2a5646c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:32.810247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffd25b-6c41-40c9-8e9d-da9a2a5646c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:32.820890 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffd25b-6c41-40c9-8e9d-da9a2a5646c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:32.822912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffd25b-6c41-40c9-8e9d-da9a2a5646c5 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:32.831597 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:32.838015 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1008/2045] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:32 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:33.059743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f46310-b6c9-4328-b0b4-374e6908dfec None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:33.061173 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-32f46310-b6c9-4328-b0b4-374e6908dfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:33.134482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f46310-b6c9-4328-b0b4-374e6908dfec None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:33.139155 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e683ad-7e0d-4c2d-966c-4ff9909ca68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:33.139783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e683ad-7e0d-4c2d-966c-4ff9909ca68b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:33.140064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e683ad-7e0d-4c2d-966c-4ff9909ca68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:33.140339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e683ad-7e0d-4c2d-966c-4ff9909ca68b admin admin] PATH_INFO: `/v3/projects/3c30f843007d4200ac64e2189003698f` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:33.141482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e683ad-7e0d-4c2d-966c-4ff9909ca68b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c30f843007d4200ac64e2189003698f)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:33.143476 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e683ad-7e0d-4c2d-966c-4ff9909ca68b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:33.288692 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-47e683ad-7e0d-4c2d-966c-4ff9909ca68b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c30f843007d4200ac64e2189003698f', 'request_id': 'req-47e683ad-7e0d-4c2d-966c-4ff9909ca68b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:33.295030 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-47e683ad-7e0d-4c2d-966c-4ff9909ca68b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c30f843007d4200ac64e2189003698f', 'request_id': 'req-47e683ad-7e0d-4c2d-966c-4ff9909ca68b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:33.382553 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffd25b-6c41-40c9-8e9d-da9a2a5646c5 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:33.391214 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffd25b-6c41-40c9-8e9d-da9a2a5646c5 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:33.441402 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbacad-0870-4be0-b6eb-8fd570f1c858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:33.442115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbacad-0870-4be0-b6eb-8fd570f1c858 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:33.442397 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbacad-0870-4be0-b6eb-8fd570f1c858 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:33.442675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbacad-0870-4be0-b6eb-8fd570f1c858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:33.444615 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b6cbacad-0870-4be0-b6eb-8fd570f1c858 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:33.458678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-47e683ad-7e0d-4c2d-966c-4ff9909ca68b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c30f843007d4200ac64e2189003698f', 'request_id': 'req-47e683ad-7e0d-4c2d-966c-4ff9909ca68b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:33.474269 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-47e683ad-7e0d-4c2d-966c-4ff9909ca68b admin admin] The token cache is being invalidate because project 3c30f843007d4200ac64e2189003698f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:33.474566 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-47e683ad-7e0d-4c2d-966c-4ff9909ca68b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:33.475327 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-47e683ad-7e0d-4c2d-966c-4ff9909ca68b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:33.476846 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1038/2046] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:33 2022] DELETE /identity/v3/projects/3c30f843007d4200ac64e2189003698f => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:33.481643 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6839e4-5b33-4315-93d0-0788665aa7ee None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:33.481949 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6839e4-5b33-4315-93d0-0788665aa7ee None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:33.482220 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6839e4-5b33-4315-93d0-0788665aa7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:33.504569 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-8b6839e4-5b33-4315-93d0-0788665aa7ee None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:33.506532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8b6839e4-5b33-4315-93d0-0788665aa7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:33.508569 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cbacad-0870-4be0-b6eb-8fd570f1c858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:33.510680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cbacad-0870-4be0-b6eb-8fd570f1c858 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:33.519952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1009/2047] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:33 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:36:33.568980 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1039/2048] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:33.576653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15975800-dd4c-40b3-b0c4-7e9912e029b4 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:33.576942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15975800-dd4c-40b3-b0c4-7e9912e029b4 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:33.579781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15975800-dd4c-40b3-b0c4-7e9912e029b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:33.598978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-15975800-dd4c-40b3-b0c4-7e9912e029b4 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:33.606769 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-15975800-dd4c-40b3-b0c4-7e9912e029b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:33.609117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1010/2049] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:33.613363 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6839e4-5b33-4315-93d0-0788665aa7ee None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:33.623163 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6839e4-5b33-4315-93d0-0788665aa7ee None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:33.628526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321af85-8fcd-4025-8642-45b79e2db33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:33.635463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321af85-8fcd-4025-8642-45b79e2db33b admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:33.635463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321af85-8fcd-4025-8642-45b79e2db33b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:33.635463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321af85-8fcd-4025-8642-45b79e2db33b admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:33.635463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321af85-8fcd-4025-8642-45b79e2db33b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:36:33.635463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321af85-8fcd-4025-8642-45b79e2db33b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:33.635463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321af85-8fcd-4025-8642-45b79e2db33b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:33.644515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1040/2050] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:36:33.649366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15975800-dd4c-40b3-b0c4-7e9912e029b4 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:33.658962 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15975800-dd4c-40b3-b0c4-7e9912e029b4 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:33.663649 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a346d5-37b4-4e57-80f4-ec82cbf31739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:33.665971 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a346d5-37b4-4e57-80f4-ec82cbf31739 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:33.665971 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a346d5-37b4-4e57-80f4-ec82cbf31739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:33.665971 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a346d5-37b4-4e57-80f4-ec82cbf31739 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:33.665971 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a346d5-37b4-4e57-80f4-ec82cbf31739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:33.667338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a346d5-37b4-4e57-80f4-ec82cbf31739 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:33.790448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1011/2051] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:33 2022] POST /identity/v3/projects => generated 405 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:33.794520 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321af85-8fcd-4025-8642-45b79e2db33b admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:33.804975 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321af85-8fcd-4025-8642-45b79e2db33b admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:33.863485 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d88342-cf39-4e56-a7a5-6b1ad60da571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:33.864111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d88342-cf39-4e56-a7a5-6b1ad60da571 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:33.864283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d88342-cf39-4e56-a7a5-6b1ad60da571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:33.864559 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d88342-cf39-4e56-a7a5-6b1ad60da571 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:33.865199 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d88342-cf39-4e56-a7a5-6b1ad60da571 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:33.867190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d88342-cf39-4e56-a7a5-6b1ad60da571 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:34.027489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1041/2052] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:33 2022] POST /identity/v3/users => generated 303 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:34.028750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a346d5-37b4-4e57-80f4-ec82cbf31739 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:34.039637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a346d5-37b4-4e57-80f4-ec82cbf31739 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:34.046737 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a5e7e-ddfa-4269-a4ad-0f3315cf4922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:34.047404 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a5e7e-ddfa-4269-a4ad-0f3315cf4922 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:34.047686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a5e7e-ddfa-4269-a4ad-0f3315cf4922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:34.047961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a5e7e-ddfa-4269-a4ad-0f3315cf4922 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:34.048607 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a5e7e-ddfa-4269-a4ad-0f3315cf4922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:34.050207 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a5e7e-ddfa-4269-a4ad-0f3315cf4922 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:34.060859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1012/2053] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:34.065570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d88342-cf39-4e56-a7a5-6b1ad60da571 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:34.075455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d88342-cf39-4e56-a7a5-6b1ad60da571 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:34.080627 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37527e91-196f-4c9a-9652-b628ff7446bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:34.081228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37527e91-196f-4c9a-9652-b628ff7446bf admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:34.081511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37527e91-196f-4c9a-9652-b628ff7446bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:34.081785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37527e91-196f-4c9a-9652-b628ff7446bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:34.082426 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37527e91-196f-4c9a-9652-b628ff7446bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:34.084059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37527e91-196f-4c9a-9652-b628ff7446bf admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:34.104690 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1042/2054] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:34.107377 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a5e7e-ddfa-4269-a4ad-0f3315cf4922 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:34.117526 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a5e7e-ddfa-4269-a4ad-0f3315cf4922 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:34.122079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebeaaf-5397-4dd5-b1c6-16c0cbc5bda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:34.122733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebeaaf-5397-4dd5-b1c6-16c0cbc5bda9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:34.123078 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebeaaf-5397-4dd5-b1c6-16c0cbc5bda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:34.123362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebeaaf-5397-4dd5-b1c6-16c0cbc5bda9 admin admin] PATH_INFO: `/v3/projects/a6d83fb65c95456d995b530844e6797c/users/56e3961aa49a430291f924cfd4c190db/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:34.138344 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebeaaf-5397-4dd5-b1c6-16c0cbc5bda9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6d83fb65c95456d995b530844e6797c, user_id=56e3961aa49a430291f924cfd4c190db, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:34.141299 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebeaaf-5397-4dd5-b1c6-16c0cbc5bda9 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:34.273980 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1013/2055] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:34 2022] PUT /identity/v3/projects/a6d83fb65c95456d995b530844e6797c/users/56e3961aa49a430291f924cfd4c190db/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:34.282541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b536ee-bcef-4ebf-a759-e30c9e3250eb None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:34.282780 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b536ee-bcef-4ebf-a759-e30c9e3250eb None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:34.283106 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b536ee-bcef-4ebf-a759-e30c9e3250eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:34.311729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-03b536ee-bcef-4ebf-a759-e30c9e3250eb None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:34.312946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-03b536ee-bcef-4ebf-a759-e30c9e3250eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:34.316276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebeaaf-5397-4dd5-b1c6-16c0cbc5bda9 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:34.320510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-13ebeaaf-5397-4dd5-b1c6-16c0cbc5bda9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:34.411889 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebeaaf-5397-4dd5-b1c6-16c0cbc5bda9 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:34.413795 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1043/2056] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:34.414976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b536ee-bcef-4ebf-a759-e30c9e3250eb None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:34.416672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-03b536ee-bcef-4ebf-a759-e30c9e3250eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:34.417624 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ad5da-9a83-4b9d-99fc-442146889515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:34.418275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad5da-9a83-4b9d-99fc-442146889515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:34.418560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad5da-9a83-4b9d-99fc-442146889515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:34.418878 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad5da-9a83-4b9d-99fc-442146889515 admin admin] PATH_INFO: `/v3/users/5ca108cd9b064d729b047d4d2417e2be` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:34.420269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ad5da-9a83-4b9d-99fc-442146889515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ca108cd9b064d729b047d4d2417e2be)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:34.423048 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ad5da-9a83-4b9d-99fc-442146889515 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:34.430777 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:34.434330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b536ee-bcef-4ebf-a759-e30c9e3250eb None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:34.451281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2abfc0-6c3c-45a4-994c-cae0e0f33179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:34.451935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2abfc0-6c3c-45a4-994c-cae0e0f33179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:34.452222 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2abfc0-6c3c-45a4-994c-cae0e0f33179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:34.453775 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2abfc0-6c3c-45a4-994c-cae0e0f33179 admin admin] PATH_INFO: `/v3/users/783e805363ee444788177ee1359dac74` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:34.455663 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2abfc0-6c3c-45a4-994c-cae0e0f33179 admin admin] RBAC: Authorizing `identity:delete_user(user_id=783e805363ee444788177ee1359dac74)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:34.457814 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2abfc0-6c3c-45a4-994c-cae0e0f33179 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:34.693951 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-616ad5da-9a83-4b9d-99fc-442146889515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ca108cd9b064d729b047d4d2417e2be'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:34.741132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-cd2abfc0-6c3c-45a4-994c-cae0e0f33179 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '783e805363ee444788177ee1359dac74'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:34.800878 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-616ad5da-9a83-4b9d-99fc-442146889515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ca108cd9b064d729b047d4d2417e2be'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:34.801508 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-616ad5da-9a83-4b9d-99fc-442146889515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ca108cd9b064d729b047d4d2417e2be'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:34.814542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-616ad5da-9a83-4b9d-99fc-442146889515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ca108cd9b064d729b047d4d2417e2be'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:34.821321 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-616ad5da-9a83-4b9d-99fc-442146889515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ca108cd9b064d729b047d4d2417e2be'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:34.827242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1014/2057] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:34 2022] DELETE /identity/v3/users/5ca108cd9b064d729b047d4d2417e2be => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:34.828021 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ad5da-9a83-4b9d-99fc-442146889515 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:34.829968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-616ad5da-9a83-4b9d-99fc-442146889515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:34.869339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-cd2abfc0-6c3c-45a4-994c-cae0e0f33179 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '783e805363ee444788177ee1359dac74'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:34.882654 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-cd2abfc0-6c3c-45a4-994c-cae0e0f33179 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '783e805363ee444788177ee1359dac74'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:34.883514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-cd2abfc0-6c3c-45a4-994c-cae0e0f33179 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '783e805363ee444788177ee1359dac74'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:34.884866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ad5da-9a83-4b9d-99fc-442146889515 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:34.888450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcc6c6-03e5-4028-a4d4-f5163774de4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:34.889089 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcc6c6-03e5-4028-a4d4-f5163774de4a service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:34.889369 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcc6c6-03e5-4028-a4d4-f5163774de4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:34.889640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcc6c6-03e5-4028-a4d4-f5163774de4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:34.890701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-cd2abfc0-6c3c-45a4-994c-cae0e0f33179 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '783e805363ee444788177ee1359dac74'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:34.892043 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f0fcc6c6-03e5-4028-a4d4-f5163774de4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:34.897311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1044/2058] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:34 2022] DELETE /identity/v3/users/783e805363ee444788177ee1359dac74 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:34.901475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64efe7-1e67-4f0c-a6aa-98d7f4ac1c44 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:34.901878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64efe7-1e67-4f0c-a6aa-98d7f4ac1c44 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:34.904255 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64efe7-1e67-4f0c-a6aa-98d7f4ac1c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:34.920435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-ae64efe7-1e67-4f0c-a6aa-98d7f4ac1c44 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:34.921746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ae64efe7-1e67-4f0c-a6aa-98d7f4ac1c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:34.955800 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcc6c6-03e5-4028-a4d4-f5163774de4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:34.955800 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcc6c6-03e5-4028-a4d4-f5163774de4a service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:34.965635 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1015/2059] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:34.966932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3f75b-856f-4bcf-aa5f-fc2e112054b7 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:34.967187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3f75b-856f-4bcf-aa5f-fc2e112054b7 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:34.967473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3f75b-856f-4bcf-aa5f-fc2e112054b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:34.980809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1045/2060] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:34.986866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-c5c3f75b-856f-4bcf-aa5f-fc2e112054b7 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:34.988146 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c5c3f75b-856f-4bcf-aa5f-fc2e112054b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:34.991702 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64efe7-1e67-4f0c-a6aa-98d7f4ac1c44 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:34.993336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ae64efe7-1e67-4f0c-a6aa-98d7f4ac1c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:34.996679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1016/2061] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:35.006685 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3f75b-856f-4bcf-aa5f-fc2e112054b7 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:35.008169 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c5c3f75b-856f-4bcf-aa5f-fc2e112054b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:35.050835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64efe7-1e67-4f0c-a6aa-98d7f4ac1c44 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:35.054503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab8a92-c025-4d7f-a08c-bf0b8b36ef64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:35.055293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab8a92-c025-4d7f-a08c-bf0b8b36ef64 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:35.055744 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab8a92-c025-4d7f-a08c-bf0b8b36ef64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:35.056033 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab8a92-c025-4d7f-a08c-bf0b8b36ef64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:35.065877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab8a92-c025-4d7f-a08c-bf0b8b36ef64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:35.067218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab8a92-c025-4d7f-a08c-bf0b8b36ef64 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:35.073516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3f75b-856f-4bcf-aa5f-fc2e112054b7 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:35.073516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd185dc-b4c8-4845-88bf-db9f7ca13a83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:35.073516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd185dc-b4c8-4845-88bf-db9f7ca13a83 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:35.073516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd185dc-b4c8-4845-88bf-db9f7ca13a83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:35.074453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd185dc-b4c8-4845-88bf-db9f7ca13a83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:35.084791 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1046/2062] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:36:35.091072 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd185dc-b4c8-4845-88bf-db9f7ca13a83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:35.092882 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd185dc-b4c8-4845-88bf-db9f7ca13a83 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:35.105746 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1017/2063] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:36.209937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab8a92-c025-4d7f-a08c-bf0b8b36ef64 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:36.211681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-2dab8a92-c025-4d7f-a08c-bf0b8b36ef64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:36.229700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:36.236537 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab8a92-c025-4d7f-a08c-bf0b8b36ef64 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:36.243285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0590163-3415-4149-b09c-6af2ea484add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:36.244226 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0590163-3415-4149-b09c-6af2ea484add admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:36.244659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0590163-3415-4149-b09c-6af2ea484add admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:36.245187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0590163-3415-4149-b09c-6af2ea484add admin admin] PATH_INFO: `/v3/projects/44b606ca920944cc9b6773fc5f0678d5` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:36.246577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0590163-3415-4149-b09c-6af2ea484add admin admin] RBAC: Authorizing `identity:delete_project(project_id=44b606ca920944cc9b6773fc5f0678d5)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:36.248907 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0590163-3415-4149-b09c-6af2ea484add admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:36.402740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd185dc-b4c8-4845-88bf-db9f7ca13a83 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:36.405316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-2dd185dc-b4c8-4845-88bf-db9f7ca13a83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:36.429083 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd185dc-b4c8-4845-88bf-db9f7ca13a83 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:36.448130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d0590163-3415-4149-b09c-6af2ea484add admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44b606ca920944cc9b6773fc5f0678d5', 'request_id': 'req-d0590163-3415-4149-b09c-6af2ea484add'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:36.467657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d0590163-3415-4149-b09c-6af2ea484add admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44b606ca920944cc9b6773fc5f0678d5', 'request_id': 'req-d0590163-3415-4149-b09c-6af2ea484add'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:36.526174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbf6b8-9894-4882-8c6b-ec58459e66fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:36.528535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbf6b8-9894-4882-8c6b-ec58459e66fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:36.529114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbf6b8-9894-4882-8c6b-ec58459e66fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:36.529566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbf6b8-9894-4882-8c6b-ec58459e66fb admin admin] PATH_INFO: `/v3/projects/ecf87234834642fb897cd7ad24ab3a19` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:36.531144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbf6b8-9894-4882-8c6b-ec58459e66fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecf87234834642fb897cd7ad24ab3a19)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:36.534693 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbf6b8-9894-4882-8c6b-ec58459e66fb admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:36.549776 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d0590163-3415-4149-b09c-6af2ea484add admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44b606ca920944cc9b6773fc5f0678d5', 'request_id': 'req-d0590163-3415-4149-b09c-6af2ea484add'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:36.571065 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d0590163-3415-4149-b09c-6af2ea484add admin admin] The token cache is being invalidate because project 44b606ca920944cc9b6773fc5f0678d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:36.571366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d0590163-3415-4149-b09c-6af2ea484add admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:36.572093 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d0590163-3415-4149-b09c-6af2ea484add admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:36.573651 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1047/2064] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:36 2022] DELETE /identity/v3/projects/44b606ca920944cc9b6773fc5f0678d5 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:36.577665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0590163-3415-4149-b09c-6af2ea484add admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:36.579269 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d0590163-3415-4149-b09c-6af2ea484add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:36.592174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0590163-3415-4149-b09c-6af2ea484add admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:36.596940 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e6b72-cf78-4a73-b573-3b14eea49765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:36.597560 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e6b72-cf78-4a73-b573-3b14eea49765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:36.597841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e6b72-cf78-4a73-b573-3b14eea49765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:36.598118 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e6b72-cf78-4a73-b573-3b14eea49765 admin admin] PATH_INFO: `/v3/users/4d1c2d3fc2e9435098ff52d45e9522d1` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:36.599275 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e6b72-cf78-4a73-b573-3b14eea49765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d1c2d3fc2e9435098ff52d45e9522d1)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:36.601190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e6b72-cf78-4a73-b573-3b14eea49765 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:36.687795 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-19dbf6b8-9894-4882-8c6b-ec58459e66fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecf87234834642fb897cd7ad24ab3a19', 'request_id': 'req-19dbf6b8-9894-4882-8c6b-ec58459e66fb'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:36.693199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-19dbf6b8-9894-4882-8c6b-ec58459e66fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecf87234834642fb897cd7ad24ab3a19', 'request_id': 'req-19dbf6b8-9894-4882-8c6b-ec58459e66fb'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:36.783222 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-19dbf6b8-9894-4882-8c6b-ec58459e66fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecf87234834642fb897cd7ad24ab3a19', 'request_id': 'req-19dbf6b8-9894-4882-8c6b-ec58459e66fb'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:36.813992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-19dbf6b8-9894-4882-8c6b-ec58459e66fb admin admin] The token cache is being invalidate because project ecf87234834642fb897cd7ad24ab3a19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:36.814284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-19dbf6b8-9894-4882-8c6b-ec58459e66fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:36.814983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-19dbf6b8-9894-4882-8c6b-ec58459e66fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:36.816535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1018/2065] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:36 2022] DELETE /identity/v3/projects/ecf87234834642fb897cd7ad24ab3a19 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:36.820412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbf6b8-9894-4882-8c6b-ec58459e66fb admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:36.821894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-19dbf6b8-9894-4882-8c6b-ec58459e66fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:36.891507 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbf6b8-9894-4882-8c6b-ec58459e66fb admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:36.895631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-7b6e6b72-cf78-4a73-b573-3b14eea49765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d1c2d3fc2e9435098ff52d45e9522d1'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:36.896460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c77186-32a8-4b73-ad13-62f58520a2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:36.897174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c77186-32a8-4b73-ad13-62f58520a2a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:36.897576 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c77186-32a8-4b73-ad13-62f58520a2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:36.897920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c77186-32a8-4b73-ad13-62f58520a2a8 admin admin] PATH_INFO: `/v3/users/bbc79451871a41b394b601db8b40f9d1` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:36.899179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c77186-32a8-4b73-ad13-62f58520a2a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbc79451871a41b394b601db8b40f9d1)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:36.901211 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c77186-32a8-4b73-ad13-62f58520a2a8 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:37.040879 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-7b6e6b72-cf78-4a73-b573-3b14eea49765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d1c2d3fc2e9435098ff52d45e9522d1'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:37.058371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-7b6e6b72-cf78-4a73-b573-3b14eea49765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d1c2d3fc2e9435098ff52d45e9522d1'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:37.059205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-7b6e6b72-cf78-4a73-b573-3b14eea49765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d1c2d3fc2e9435098ff52d45e9522d1'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:37.066148 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-7b6e6b72-cf78-4a73-b573-3b14eea49765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d1c2d3fc2e9435098ff52d45e9522d1'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:37.072105 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1048/2066] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:36 2022] DELETE /identity/v3/users/4d1c2d3fc2e9435098ff52d45e9522d1 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:37.073624 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34935520-e57e-4eca-b537-9bc5d06f0038 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:37.073944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34935520-e57e-4eca-b537-9bc5d06f0038 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:37.074217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34935520-e57e-4eca-b537-9bc5d06f0038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:37.115102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-34935520-e57e-4eca-b537-9bc5d06f0038 None None] MFA Rules not processed for user `56e3961aa49a430291f924cfd4c190db`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:37.117545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-34935520-e57e-4eca-b537-9bc5d06f0038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:37.231833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1049/2067] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:37 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:37.236171 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34935520-e57e-4eca-b537-9bc5d06f0038 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:37.237478 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-34935520-e57e-4eca-b537-9bc5d06f0038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:37.304761 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34935520-e57e-4eca-b537-9bc5d06f0038 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:37.310327 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4356461-9cb8-40f3-9ed0-cdd4a70662d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:37.311098 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4356461-9cb8-40f3-9ed0-cdd4a70662d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:37.311487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4356461-9cb8-40f3-9ed0-cdd4a70662d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:37.313842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-10c77186-32a8-4b73-ad13-62f58520a2a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbc79451871a41b394b601db8b40f9d1'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:37.315083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4356461-9cb8-40f3-9ed0-cdd4a70662d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:37.316377 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4356461-9cb8-40f3-9ed0-cdd4a70662d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:37.319265 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4356461-9cb8-40f3-9ed0-cdd4a70662d3 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:37.422600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-10c77186-32a8-4b73-ad13-62f58520a2a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbc79451871a41b394b601db8b40f9d1'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:37.423282 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-10c77186-32a8-4b73-ad13-62f58520a2a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbc79451871a41b394b601db8b40f9d1'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:37.438476 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-10c77186-32a8-4b73-ad13-62f58520a2a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbc79451871a41b394b601db8b40f9d1'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:37.445396 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-10c77186-32a8-4b73-ad13-62f58520a2a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbc79451871a41b394b601db8b40f9d1'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:37.452300 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1019/2068] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:36 2022] DELETE /identity/v3/users/bbc79451871a41b394b601db8b40f9d1 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:37.466937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1050/2069] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:37 2022] POST /identity/v3/projects => generated 405 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:36:37.468091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c77186-32a8-4b73-ad13-62f58520a2a8 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:37.469560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-10c77186-32a8-4b73-ad13-62f58520a2a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:37.542835 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c77186-32a8-4b73-ad13-62f58520a2a8 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:37.582225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:37.602691 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28397d46-b037-49f0-a26b-f70fda6aea0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:37.603353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28397d46-b037-49f0-a26b-f70fda6aea0b admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:37.603637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28397d46-b037-49f0-a26b-f70fda6aea0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:37.603913 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28397d46-b037-49f0-a26b-f70fda6aea0b admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:37.604583 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28397d46-b037-49f0-a26b-f70fda6aea0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:37.606568 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28397d46-b037-49f0-a26b-f70fda6aea0b admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:37.659342 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4356461-9cb8-40f3-9ed0-cdd4a70662d3 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:37.661244 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f4356461-9cb8-40f3-9ed0-cdd4a70662d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:37.685007 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4356461-9cb8-40f3-9ed0-cdd4a70662d3 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:37.691174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d594dcd-19c0-4720-8faf-3329097c53fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:37.691900 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d594dcd-19c0-4720-8faf-3329097c53fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:37.692288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d594dcd-19c0-4720-8faf-3329097c53fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:37.692639 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d594dcd-19c0-4720-8faf-3329097c53fa admin admin] PATH_INFO: `/v3/projects/3e94be89fdd144f0b1ea241471e7cbf8` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:37.693873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d594dcd-19c0-4720-8faf-3329097c53fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e94be89fdd144f0b1ea241471e7cbf8)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:37.695963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d594dcd-19c0-4720-8faf-3329097c53fa admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:37.761223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1020/2070] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:37 2022] POST /identity/v3/users => generated 309 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:37.765036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28397d46-b037-49f0-a26b-f70fda6aea0b admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:37.786468 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28397d46-b037-49f0-a26b-f70fda6aea0b admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:37.792383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a352a19-bf81-404b-bf3b-54a2c3c56de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:37.806191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a352a19-bf81-404b-bf3b-54a2c3c56de5 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:37.806191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a352a19-bf81-404b-bf3b-54a2c3c56de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:37.806191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a352a19-bf81-404b-bf3b-54a2c3c56de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:37.806191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a352a19-bf81-404b-bf3b-54a2c3c56de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:37.806191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a352a19-bf81-404b-bf3b-54a2c3c56de5 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:37.807029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1021/2071] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:37.811099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a352a19-bf81-404b-bf3b-54a2c3c56de5 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:37.821365 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a352a19-bf81-404b-bf3b-54a2c3c56de5 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:37.826428 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e044d-7bc6-4755-a20a-3c9ba92f286a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:37.827120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e044d-7bc6-4755-a20a-3c9ba92f286a admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:37.827405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e044d-7bc6-4755-a20a-3c9ba92f286a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:37.827689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e044d-7bc6-4755-a20a-3c9ba92f286a admin admin] PATH_INFO: `/v3/projects/ec8ee3a8020e499189c10f89d19acafd/users/3df1a36fe2c74e6c9b0e7449e48b0549/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:37.843509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e044d-7bc6-4755-a20a-3c9ba92f286a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8ee3a8020e499189c10f89d19acafd, user_id=3df1a36fe2c74e6c9b0e7449e48b0549, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:37.853120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e044d-7bc6-4755-a20a-3c9ba92f286a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:37.954612 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1022/2072] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:37 2022] PUT /identity/v3/projects/ec8ee3a8020e499189c10f89d19acafd/users/3df1a36fe2c74e6c9b0e7449e48b0549/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:37.958406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e044d-7bc6-4755-a20a-3c9ba92f286a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:37.962625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-9d594dcd-19c0-4720-8faf-3329097c53fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e94be89fdd144f0b1ea241471e7cbf8', 'request_id': 'req-9d594dcd-19c0-4720-8faf-3329097c53fa'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:37.970216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-9d594dcd-19c0-4720-8faf-3329097c53fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e94be89fdd144f0b1ea241471e7cbf8', 'request_id': 'req-9d594dcd-19c0-4720-8faf-3329097c53fa'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:38.026596 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e044d-7bc6-4755-a20a-3c9ba92f286a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:38.081942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5a738-b385-4794-849e-33b08dd2d308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:38.082475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5a738-b385-4794-849e-33b08dd2d308 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:38.082753 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5a738-b385-4794-849e-33b08dd2d308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:38.083181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5a738-b385-4794-849e-33b08dd2d308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:38.083854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5a738-b385-4794-849e-33b08dd2d308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:38.084760 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-9d594dcd-19c0-4720-8faf-3329097c53fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e94be89fdd144f0b1ea241471e7cbf8', 'request_id': 'req-9d594dcd-19c0-4720-8faf-3329097c53fa'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:38.085445 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5a738-b385-4794-849e-33b08dd2d308 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:38.097674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-9d594dcd-19c0-4720-8faf-3329097c53fa admin admin] The token cache is being invalidate because project 3e94be89fdd144f0b1ea241471e7cbf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:38.099981 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-9d594dcd-19c0-4720-8faf-3329097c53fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:38.100089 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1023/2073] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:38.100178 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-9d594dcd-19c0-4720-8faf-3329097c53fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:38.100313 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5a738-b385-4794-849e-33b08dd2d308 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:38.101389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1051/2074] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:37 2022] DELETE /identity/v3/projects/3e94be89fdd144f0b1ea241471e7cbf8 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:38.101955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ffe5a738-b385-4794-849e-33b08dd2d308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:38.106362 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d594dcd-19c0-4720-8faf-3329097c53fa admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:38.107808 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-9d594dcd-19c0-4720-8faf-3329097c53fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:38.115040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5a738-b385-4794-849e-33b08dd2d308 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:38.119986 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a12dc9-a473-442a-90b4-f9dcd07cd35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:38.120614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a12dc9-a473-442a-90b4-f9dcd07cd35a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:38.120894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a12dc9-a473-442a-90b4-f9dcd07cd35a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:38.121166 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a12dc9-a473-442a-90b4-f9dcd07cd35a admin admin] PATH_INFO: `/v3/projects/f5cd7ab190084a629de746b67ba1c164` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:38.121288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d594dcd-19c0-4720-8faf-3329097c53fa admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:38.122287 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a12dc9-a473-442a-90b4-f9dcd07cd35a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5cd7ab190084a629de746b67ba1c164)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:38.124236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a12dc9-a473-442a-90b4-f9dcd07cd35a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:38.126746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299869de-19e0-40ba-b9d4-29ec7bbaf64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:38.127437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299869de-19e0-40ba-b9d4-29ec7bbaf64d admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:38.127740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299869de-19e0-40ba-b9d4-29ec7bbaf64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:38.128017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299869de-19e0-40ba-b9d4-29ec7bbaf64d admin admin] PATH_INFO: `/v3/projects/ec8ee3a8020e499189c10f89d19acafd/users/3df1a36fe2c74e6c9b0e7449e48b0549/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:38.131176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299869de-19e0-40ba-b9d4-29ec7bbaf64d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8ee3a8020e499189c10f89d19acafd, user_id=3df1a36fe2c74e6c9b0e7449e48b0549, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:38.134035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299869de-19e0-40ba-b9d4-29ec7bbaf64d admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:38.148978 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1052/2075] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:38 2022] PUT /identity/v3/projects/ec8ee3a8020e499189c10f89d19acafd/users/3df1a36fe2c74e6c9b0e7449e48b0549/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:38.152707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d826c-a17d-4d6f-9b0a-6e36a36e1a7f None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:38.152707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d826c-a17d-4d6f-9b0a-6e36a36e1a7f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:38.152707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d826c-a17d-4d6f-9b0a-6e36a36e1a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:38.171025 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-1c6d826c-a17d-4d6f-9b0a-6e36a36e1a7f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:38.172811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-1c6d826c-a17d-4d6f-9b0a-6e36a36e1a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:38.233141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1053/2076] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:38.241753 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397e831-8505-4c95-8ba1-7452c81b4894 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:38.242045 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397e831-8505-4c95-8ba1-7452c81b4894 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:38.242319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397e831-8505-4c95-8ba1-7452c81b4894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:38.259375 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-48a12dc9-a473-442a-90b4-f9dcd07cd35a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5cd7ab190084a629de746b67ba1c164', 'request_id': 'req-48a12dc9-a473-442a-90b4-f9dcd07cd35a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:38.261494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-c397e831-8505-4c95-8ba1-7452c81b4894 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:38.262891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c397e831-8505-4c95-8ba1-7452c81b4894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:38.265176 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-48a12dc9-a473-442a-90b4-f9dcd07cd35a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5cd7ab190084a629de746b67ba1c164', 'request_id': 'req-48a12dc9-a473-442a-90b4-f9dcd07cd35a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:38.326183 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1054/2077] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:38.330593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397e831-8505-4c95-8ba1-7452c81b4894 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:38.340206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397e831-8505-4c95-8ba1-7452c81b4894 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:38.346300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f9f41-6fa6-49d7-a7f0-6d9460619eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:38.346971 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f9f41-6fa6-49d7-a7f0-6d9460619eac admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:38.347511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f9f41-6fa6-49d7-a7f0-6d9460619eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:38.347511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f9f41-6fa6-49d7-a7f0-6d9460619eac admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:38.348157 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f9f41-6fa6-49d7-a7f0-6d9460619eac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:36:38.348463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f9f41-6fa6-49d7-a7f0-6d9460619eac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:38.350059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f9f41-6fa6-49d7-a7f0-6d9460619eac admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:38.364785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1055/2078] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:36:38.368821 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f9f41-6fa6-49d7-a7f0-6d9460619eac admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:38.377847 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-48a12dc9-a473-442a-90b4-f9dcd07cd35a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5cd7ab190084a629de746b67ba1c164', 'request_id': 'req-48a12dc9-a473-442a-90b4-f9dcd07cd35a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:38.389364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f9f41-6fa6-49d7-a7f0-6d9460619eac admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:38.390317 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-48a12dc9-a473-442a-90b4-f9dcd07cd35a admin admin] The token cache is being invalidate because project f5cd7ab190084a629de746b67ba1c164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:38.390612 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-48a12dc9-a473-442a-90b4-f9dcd07cd35a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:38.391363 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-48a12dc9-a473-442a-90b4-f9dcd07cd35a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:38.393208 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1024/2079] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:38 2022] DELETE /identity/v3/projects/f5cd7ab190084a629de746b67ba1c164 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:38.394559 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10278957-5a38-4e94-9a87-c9a8471b0a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:38.395216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10278957-5a38-4e94-9a87-c9a8471b0a28 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:38.395498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10278957-5a38-4e94-9a87-c9a8471b0a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:38.395872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10278957-5a38-4e94-9a87-c9a8471b0a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:38.396511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10278957-5a38-4e94-9a87-c9a8471b0a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:38.397165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a12dc9-a473-442a-90b4-f9dcd07cd35a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:38.398423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10278957-5a38-4e94-9a87-c9a8471b0a28 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:38.399538 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-48a12dc9-a473-442a-90b4-f9dcd07cd35a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:38.418413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a12dc9-a473-442a-90b4-f9dcd07cd35a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:38.424060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496b733-ce59-43b0-876f-24d75d845121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:38.424702 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496b733-ce59-43b0-876f-24d75d845121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:38.424985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496b733-ce59-43b0-876f-24d75d845121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:38.425258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496b733-ce59-43b0-876f-24d75d845121 admin admin] PATH_INFO: `/v3/users/4ff900a6e727483885a4496e86d065ec` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:38.426402 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496b733-ce59-43b0-876f-24d75d845121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ff900a6e727483885a4496e86d065ec)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:38.428557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496b733-ce59-43b0-876f-24d75d845121 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:38.508465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1056/2080] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:38 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:36:38.509394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10278957-5a38-4e94-9a87-c9a8471b0a28 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:38.511034 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-10278957-5a38-4e94-9a87-c9a8471b0a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:38.584027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10278957-5a38-4e94-9a87-c9a8471b0a28 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:38.589065 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08d5ef-1a1f-4b69-a5bf-99d3ddce98db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:38.589681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08d5ef-1a1f-4b69-a5bf-99d3ddce98db admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:38.589961 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08d5ef-1a1f-4b69-a5bf-99d3ddce98db admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:38.590546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08d5ef-1a1f-4b69-a5bf-99d3ddce98db admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:38.591157 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08d5ef-1a1f-4b69-a5bf-99d3ddce98db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:38.593085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08d5ef-1a1f-4b69-a5bf-99d3ddce98db admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:38.681007 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1057/2081] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:38 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:38.685120 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08d5ef-1a1f-4b69-a5bf-99d3ddce98db admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:38.689494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8496b733-ce59-43b0-876f-24d75d845121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ff900a6e727483885a4496e86d065ec'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:38.695031 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08d5ef-1a1f-4b69-a5bf-99d3ddce98db admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:38.699642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9d8da-1980-49a0-adf1-a05370945ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:38.700212 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9d8da-1980-49a0-adf1-a05370945ebc admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:38.700490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9d8da-1980-49a0-adf1-a05370945ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:38.700764 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9d8da-1980-49a0-adf1-a05370945ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:38.701393 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9d8da-1980-49a0-adf1-a05370945ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:38.703719 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9d8da-1980-49a0-adf1-a05370945ebc admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:38.714148 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1058/2082] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:38.720656 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9d8da-1980-49a0-adf1-a05370945ebc admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:38.729346 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9d8da-1980-49a0-adf1-a05370945ebc admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:38.734245 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a7906-c50e-4d52-a35a-25510d956fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:38.735145 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a7906-c50e-4d52-a35a-25510d956fef admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:38.735426 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a7906-c50e-4d52-a35a-25510d956fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:38.735714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a7906-c50e-4d52-a35a-25510d956fef admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:38.736830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a7906-c50e-4d52-a35a-25510d956fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:38.738073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a7906-c50e-4d52-a35a-25510d956fef admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:38.750374 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1059/2083] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:38.754387 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a7906-c50e-4d52-a35a-25510d956fef admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:38.764007 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a7906-c50e-4d52-a35a-25510d956fef admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:38.767093 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8496b733-ce59-43b0-876f-24d75d845121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ff900a6e727483885a4496e86d065ec'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:38.767788 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8496b733-ce59-43b0-876f-24d75d845121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ff900a6e727483885a4496e86d065ec'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:38.770288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92977a6c-72ee-43f1-8ef7-b3c886b76134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:38.772318 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92977a6c-72ee-43f1-8ef7-b3c886b76134 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:38.772318 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92977a6c-72ee-43f1-8ef7-b3c886b76134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:38.772318 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92977a6c-72ee-43f1-8ef7-b3c886b76134 admin admin] PATH_INFO: `/v3/projects/790e800eecf941f6a824581d3fa4f91f/users/0e1ce8fc20334659a310c045037c1525/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:38.787003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8496b733-ce59-43b0-876f-24d75d845121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ff900a6e727483885a4496e86d065ec'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:38.794886 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92977a6c-72ee-43f1-8ef7-b3c886b76134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=790e800eecf941f6a824581d3fa4f91f, user_id=0e1ce8fc20334659a310c045037c1525, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:38.796137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8496b733-ce59-43b0-876f-24d75d845121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ff900a6e727483885a4496e86d065ec'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:38.798410 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92977a6c-72ee-43f1-8ef7-b3c886b76134 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:38.803137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1025/2084] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:38 2022] DELETE /identity/v3/users/4ff900a6e727483885a4496e86d065ec => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:38.934882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1060/2085] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:38 2022] PUT /identity/v3/projects/790e800eecf941f6a824581d3fa4f91f/users/0e1ce8fc20334659a310c045037c1525/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:38.939089 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5892054-7881-411e-9201-51ee63662baa None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:38.939400 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5892054-7881-411e-9201-51ee63662baa None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:38.939680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5892054-7881-411e-9201-51ee63662baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:38.968135 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-b5892054-7881-411e-9201-51ee63662baa None None] MFA Rules not processed for user `0e1ce8fc20334659a310c045037c1525`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:38.969840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b5892054-7881-411e-9201-51ee63662baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:39.035340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1026/2086] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:39.046852 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92977a6c-72ee-43f1-8ef7-b3c886b76134 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:39.049271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-92977a6c-72ee-43f1-8ef7-b3c886b76134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:39.131860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92977a6c-72ee-43f1-8ef7-b3c886b76134 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:39.135804 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5eae4-5fa0-4d33-b0e4-1dfaedc37138 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:39.136421 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5eae4-5fa0-4d33-b0e4-1dfaedc37138 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:39.138248 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5eae4-5fa0-4d33-b0e4-1dfaedc37138 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:39.138248 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5eae4-5fa0-4d33-b0e4-1dfaedc37138 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:39.148194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5eae4-5fa0-4d33-b0e4-1dfaedc37138 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:39.149177 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5eae4-5fa0-4d33-b0e4-1dfaedc37138 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:39.159567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1061/2087] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:39.179825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5892054-7881-411e-9201-51ee63662baa None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:39.181340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b5892054-7881-411e-9201-51ee63662baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:39.254357 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5892054-7881-411e-9201-51ee63662baa None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:39.259202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691abf0-5e6b-4f98-8ae0-c62966e54b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:39.259856 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691abf0-5e6b-4f98-8ae0-c62966e54b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:39.260141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691abf0-5e6b-4f98-8ae0-c62966e54b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:39.261081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691abf0-5e6b-4f98-8ae0-c62966e54b2d admin admin] PATH_INFO: `/v3/users/0e1ce8fc20334659a310c045037c1525` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:39.262234 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691abf0-5e6b-4f98-8ae0-c62966e54b2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e1ce8fc20334659a310c045037c1525)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:39.264381 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691abf0-5e6b-4f98-8ae0-c62966e54b2d admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:39.367784 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5eae4-5fa0-4d33-b0e4-1dfaedc37138 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:39.369940 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-2fb5eae4-5fa0-4d33-b0e4-1dfaedc37138 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:39.386055 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5eae4-5fa0-4d33-b0e4-1dfaedc37138 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:39.390621 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d089fc-716b-436a-bb89-e80252b83577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:39.391933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d089fc-716b-436a-bb89-e80252b83577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:39.392225 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d089fc-716b-436a-bb89-e80252b83577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:39.392534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d089fc-716b-436a-bb89-e80252b83577 admin admin] PATH_INFO: `/v3/projects/6f75ae3652a84a188b33fb15a6f0bad8` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:39.393670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d089fc-716b-436a-bb89-e80252b83577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f75ae3652a84a188b33fb15a6f0bad8)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:39.395628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d089fc-716b-436a-bb89-e80252b83577 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:39.509900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3691abf0-5e6b-4f98-8ae0-c62966e54b2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e1ce8fc20334659a310c045037c1525'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:39.588511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-28d089fc-716b-436a-bb89-e80252b83577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f75ae3652a84a188b33fb15a6f0bad8', 'request_id': 'req-28d089fc-716b-436a-bb89-e80252b83577'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:39.588511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-28d089fc-716b-436a-bb89-e80252b83577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f75ae3652a84a188b33fb15a6f0bad8', 'request_id': 'req-28d089fc-716b-436a-bb89-e80252b83577'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:39.793512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3691abf0-5e6b-4f98-8ae0-c62966e54b2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e1ce8fc20334659a310c045037c1525'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:39.794279 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3691abf0-5e6b-4f98-8ae0-c62966e54b2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e1ce8fc20334659a310c045037c1525'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:39.807153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3691abf0-5e6b-4f98-8ae0-c62966e54b2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e1ce8fc20334659a310c045037c1525'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:39.814563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3691abf0-5e6b-4f98-8ae0-c62966e54b2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e1ce8fc20334659a310c045037c1525'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:39.820536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1027/2088] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:39 2022] DELETE /identity/v3/users/0e1ce8fc20334659a310c045037c1525 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:39.824713 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c30a1-edbc-4282-ae00-a4aaab03fdff None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:39.825007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c30a1-edbc-4282-ae00-a4aaab03fdff None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:39.825294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c30a1-edbc-4282-ae00-a4aaab03fdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:39.845368 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-e06c30a1-edbc-4282-ae00-a4aaab03fdff None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:39.846539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e06c30a1-edbc-4282-ae00-a4aaab03fdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:39.905770 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1028/2089] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:39.907102 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfa046-d662-40ee-9ef5-d5115d0fd087 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:39.907387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfa046-d662-40ee-9ef5-d5115d0fd087 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:39.907697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfa046-d662-40ee-9ef5-d5115d0fd087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:39.934029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-a4cfa046-d662-40ee-9ef5-d5115d0fd087 None None] MFA Rules not processed for user `3df1a36fe2c74e6c9b0e7449e48b0549`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:39.935261 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a4cfa046-d662-40ee-9ef5-d5115d0fd087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:40.001047 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1029/2090] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:39 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:40.001881 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cfa046-d662-40ee-9ef5-d5115d0fd087 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:40.003138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a4cfa046-d662-40ee-9ef5-d5115d0fd087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:40.023609 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-28d089fc-716b-436a-bb89-e80252b83577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f75ae3652a84a188b33fb15a6f0bad8', 'request_id': 'req-28d089fc-716b-436a-bb89-e80252b83577'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:40.043375 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-28d089fc-716b-436a-bb89-e80252b83577 admin admin] The token cache is being invalidate because project 6f75ae3652a84a188b33fb15a6f0bad8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:40.043375 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-28d089fc-716b-436a-bb89-e80252b83577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:40.043375 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-28d089fc-716b-436a-bb89-e80252b83577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:40.043375 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1062/2091] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:39 2022] DELETE /identity/v3/projects/6f75ae3652a84a188b33fb15a6f0bad8 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:40.045907 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e85d1f-a78d-48e6-bb00-7f57e4977cc1 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:40.046199 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e85d1f-a78d-48e6-bb00-7f57e4977cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:40.046476 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e85d1f-a78d-48e6-bb00-7f57e4977cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:40.074201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cfa046-d662-40ee-9ef5-d5115d0fd087 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:40.077036 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:40.078885 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b46381-f1cb-4f03-bec6-ddbc5b558f66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:40.079577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b46381-f1cb-4f03-bec6-ddbc5b558f66 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:40.079915 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b46381-f1cb-4f03-bec6-ddbc5b558f66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:40.088252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b46381-f1cb-4f03-bec6-ddbc5b558f66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:40.088369 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-30e85d1f-a78d-48e6-bb00-7f57e4977cc1 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:40.088451 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-71b46381-f1cb-4f03-bec6-ddbc5b558f66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:40.088526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-30e85d1f-a78d-48e6-bb00-7f57e4977cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:40.091709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1063/2092] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:40.099222 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73596df6-488c-45cc-9854-f07097f208b3 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:40.099664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b46381-f1cb-4f03-bec6-ddbc5b558f66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:40.099932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73596df6-488c-45cc-9854-f07097f208b3 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:40.100411 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73596df6-488c-45cc-9854-f07097f208b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:40.101509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b46381-f1cb-4f03-bec6-ddbc5b558f66 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:40.113373 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1030/2093] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:40.130716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-73596df6-488c-45cc-9854-f07097f208b3 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:40.132487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-73596df6-488c-45cc-9854-f07097f208b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:40.147648 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b46381-f1cb-4f03-bec6-ddbc5b558f66 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:40.150357 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1064/2094] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:40.161260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b46381-f1cb-4f03-bec6-ddbc5b558f66 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:40.163792 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36583f17-c143-48d7-8145-bb6a9205c8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:40.164406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36583f17-c143-48d7-8145-bb6a9205c8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:40.164691 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36583f17-c143-48d7-8145-bb6a9205c8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:40.165120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36583f17-c143-48d7-8145-bb6a9205c8f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:40.165778 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36583f17-c143-48d7-8145-bb6a9205c8f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:36:40.166083 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36583f17-c143-48d7-8145-bb6a9205c8f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:40.167757 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36583f17-c143-48d7-8145-bb6a9205c8f6 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:40.184236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1031/2095] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:36:40.185703 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73596df6-488c-45cc-9854-f07097f208b3 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:40.202171 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73596df6-488c-45cc-9854-f07097f208b3 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:40.209684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590040a5-d59c-461f-aaa2-aa591071a5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:40.210371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590040a5-d59c-461f-aaa2-aa591071a5a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:40.210655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590040a5-d59c-461f-aaa2-aa591071a5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:40.211382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590040a5-d59c-461f-aaa2-aa591071a5a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:40.212034 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590040a5-d59c-461f-aaa2-aa591071a5a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:40.214083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590040a5-d59c-461f-aaa2-aa591071a5a2 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:40.377637 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1065/2096] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:40 2022] POST /identity/v3/projects => generated 371 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:36:40.381898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36583f17-c143-48d7-8145-bb6a9205c8f6 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:40.395888 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36583f17-c143-48d7-8145-bb6a9205c8f6 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:40.482932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d9482-e165-4a47-afc4-7b24cb62204a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:40.483791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d9482-e165-4a47-afc4-7b24cb62204a admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:40.484165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d9482-e165-4a47-afc4-7b24cb62204a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:40.485227 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d9482-e165-4a47-afc4-7b24cb62204a admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:40.485983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d9482-e165-4a47-afc4-7b24cb62204a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:40.488232 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d9482-e165-4a47-afc4-7b24cb62204a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:40.587078 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1032/2097] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:40 2022] POST /identity/v3/users => generated 286 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:40.588403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590040a5-d59c-461f-aaa2-aa591071a5a2 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:40.599675 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590040a5-d59c-461f-aaa2-aa591071a5a2 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:40.604318 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af9829-bbc2-41c5-9b08-61f4f2636570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:40.604920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af9829-bbc2-41c5-9b08-61f4f2636570 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:40.605317 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af9829-bbc2-41c5-9b08-61f4f2636570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:40.605596 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af9829-bbc2-41c5-9b08-61f4f2636570 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:40.606242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af9829-bbc2-41c5-9b08-61f4f2636570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:40.607861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af9829-bbc2-41c5-9b08-61f4f2636570 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:40.618395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1066/2098] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:40.622755 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d9482-e165-4a47-afc4-7b24cb62204a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:40.633078 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d9482-e165-4a47-afc4-7b24cb62204a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:40.637867 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f4bb2-5bd3-4878-a48f-c703ca89cb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:40.638535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f4bb2-5bd3-4878-a48f-c703ca89cb59 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:40.638916 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f4bb2-5bd3-4878-a48f-c703ca89cb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:40.639255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f4bb2-5bd3-4878-a48f-c703ca89cb59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:40.639901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f4bb2-5bd3-4878-a48f-c703ca89cb59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:40.642298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f4bb2-5bd3-4878-a48f-c703ca89cb59 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:40.653105 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1033/2099] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:40.657411 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af9829-bbc2-41c5-9b08-61f4f2636570 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:40.667654 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af9829-bbc2-41c5-9b08-61f4f2636570 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:40.672334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e05ed7-f2aa-4cd4-b5e2-8b230134eb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:40.672997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e05ed7-f2aa-4cd4-b5e2-8b230134eb78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:40.673352 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e05ed7-f2aa-4cd4-b5e2-8b230134eb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:40.673699 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e05ed7-f2aa-4cd4-b5e2-8b230134eb78 admin admin] PATH_INFO: `/v3/projects/a837cc5e269446f597dc6b8c44dacf39/users/2222fcfe1f5c4dc5a3c3f08c63d301b1/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:40.688470 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e05ed7-f2aa-4cd4-b5e2-8b230134eb78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a837cc5e269446f597dc6b8c44dacf39, user_id=2222fcfe1f5c4dc5a3c3f08c63d301b1, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:40.691176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e05ed7-f2aa-4cd4-b5e2-8b230134eb78 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:40.772553 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:40.789496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affed4d1-025c-4ebd-890a-4e722fc15afd None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:40.790176 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affed4d1-025c-4ebd-890a-4e722fc15afd None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:40.790635 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affed4d1-025c-4ebd-890a-4e722fc15afd None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:40.792189 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1034/2100] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 5 14:36:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:36:40.800594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf478c5d-1517-4565-b72f-d83ea718878e None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:40.800881 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf478c5d-1517-4565-b72f-d83ea718878e None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:40.801163 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf478c5d-1517-4565-b72f-d83ea718878e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:40.825351 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-bf478c5d-1517-4565-b72f-d83ea718878e None None] MFA Rules not processed for user `267825dfb0324be0910203cd5abab377`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:40.826691 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-bf478c5d-1517-4565-b72f-d83ea718878e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:40.851352 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1067/2101] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:40 2022] PUT /identity/v3/projects/a837cc5e269446f597dc6b8c44dacf39/users/2222fcfe1f5c4dc5a3c3f08c63d301b1/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:40.855055 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18642117-c069-438f-908b-552e42ac5df1 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:40.855348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18642117-c069-438f-908b-552e42ac5df1 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:40.855628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18642117-c069-438f-908b-552e42ac5df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:40.889488 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-18642117-c069-438f-908b-552e42ac5df1 None None] MFA Rules not processed for user `2222fcfe1f5c4dc5a3c3f08c63d301b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:40.891156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-18642117-c069-438f-908b-552e42ac5df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:40.936534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1035/2102] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 5 14:36:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:40.937687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf478c5d-1517-4565-b72f-d83ea718878e None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:40.939765 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-bf478c5d-1517-4565-b72f-d83ea718878e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:40.955464 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf478c5d-1517-4565-b72f-d83ea718878e None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:40.981566 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1068/2103] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:40.983420 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff48706-15e4-4a7a-9133-ff4959ce7d25 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:40.983862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff48706-15e4-4a7a-9133-ff4959ce7d25 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:40.984315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff48706-15e4-4a7a-9133-ff4959ce7d25 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:40.988849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1069/2104] 10.222.0.15 () {54 vars in 961 bytes} [Mon Sep 5 14:36:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:36:40.994054 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d92c95-0895-4587-b611-564cbe66fcae None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:40.994347 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d92c95-0895-4587-b611-564cbe66fcae None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:40.994631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d92c95-0895-4587-b611-564cbe66fcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:41.016005 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-10d92c95-0895-4587-b611-564cbe66fcae None None] MFA Rules not processed for user `267825dfb0324be0910203cd5abab377`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:41.017278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-10d92c95-0895-4587-b611-564cbe66fcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:41.032166 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353054e-971e-4423-ae26-d05a680af6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:41.032838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353054e-971e-4423-ae26-d05a680af6f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:41.033125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353054e-971e-4423-ae26-d05a680af6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:41.033424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353054e-971e-4423-ae26-d05a680af6f2 admin admin] PATH_INFO: `/v3/projects/790e800eecf941f6a824581d3fa4f91f` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:41.034647 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9353054e-971e-4423-ae26-d05a680af6f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=790e800eecf941f6a824581d3fa4f91f)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:41.036640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9353054e-971e-4423-ae26-d05a680af6f2 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:41.087021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1070/2105] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Sep 5 14:36:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:41.087954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d92c95-0895-4587-b611-564cbe66fcae None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:41.089296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-10d92c95-0895-4587-b611-564cbe66fcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:41.167101 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d92c95-0895-4587-b611-564cbe66fcae None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:41.171372 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19b043-909d-447e-bf09-33eaa5415a51 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '735986478c354ed196c601f12f3ac377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:41.172636 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19b043-909d-447e-bf09-33eaa5415a51 service glance] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:41.172919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19b043-909d-447e-bf09-33eaa5415a51 service glance] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:41.173192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19b043-909d-447e-bf09-33eaa5415a51 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:41.181736 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19b043-909d-447e-bf09-33eaa5415a51 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:41.183146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19b043-909d-447e-bf09-33eaa5415a51 service glance] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:41.191968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1071/2106] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:41.192824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19b043-909d-447e-bf09-33eaa5415a51 service glance] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:41.204222 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19b043-909d-447e-bf09-33eaa5415a51 service glance] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:41.209913 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387c476-6503-4393-b657-c59367e1fdbd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:41.210812 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387c476-6503-4393-b657-c59367e1fdbd service placement] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:41.211106 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387c476-6503-4393-b657-c59367e1fdbd service placement] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:41.211439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387c476-6503-4393-b657-c59367e1fdbd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:41.221216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387c476-6503-4393-b657-c59367e1fdbd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:41.222930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387c476-6503-4393-b657-c59367e1fdbd service placement] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:41.232581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1072/2107] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:41.364511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9353054e-971e-4423-ae26-d05a680af6f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '790e800eecf941f6a824581d3fa4f91f', 'request_id': 'req-9353054e-971e-4423-ae26-d05a680af6f2'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:41.374140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9353054e-971e-4423-ae26-d05a680af6f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '790e800eecf941f6a824581d3fa4f91f', 'request_id': 'req-9353054e-971e-4423-ae26-d05a680af6f2'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:41.497671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9353054e-971e-4423-ae26-d05a680af6f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '790e800eecf941f6a824581d3fa4f91f', 'request_id': 'req-9353054e-971e-4423-ae26-d05a680af6f2'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:41.514752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9353054e-971e-4423-ae26-d05a680af6f2 admin admin] The token cache is being invalidate because project 790e800eecf941f6a824581d3fa4f91f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:41.517283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9353054e-971e-4423-ae26-d05a680af6f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:41.519051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9353054e-971e-4423-ae26-d05a680af6f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:41.522169 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1036/2108] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:40 2022] DELETE /identity/v3/projects/790e800eecf941f6a824581d3fa4f91f => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:41.527092 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df7601-b02b-4707-8cd9-980676e26dee None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:41.527832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df7601-b02b-4707-8cd9-980676e26dee None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:41.528109 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df7601-b02b-4707-8cd9-980676e26dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:41.548549 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-60df7601-b02b-4707-8cd9-980676e26dee None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:41.549807 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-60df7601-b02b-4707-8cd9-980676e26dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:41.610465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1073/2109] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:41.619273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a6e3b-3f36-4651-aa22-a9e03c9daa9d None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:41.619739 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a6e3b-3f36-4651-aa22-a9e03c9daa9d None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:41.619965 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a6e3b-3f36-4651-aa22-a9e03c9daa9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:41.642830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-b58a6e3b-3f36-4651-aa22-a9e03c9daa9d None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:41.644099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b58a6e3b-3f36-4651-aa22-a9e03c9daa9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:41.653362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1037/2110] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:41.657059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df7601-b02b-4707-8cd9-980676e26dee None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:41.667110 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df7601-b02b-4707-8cd9-980676e26dee None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:41.672332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb98b7-c6af-43ca-9b20-6f5568344478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:41.673058 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb98b7-c6af-43ca-9b20-6f5568344478 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:41.673400 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb98b7-c6af-43ca-9b20-6f5568344478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:41.673784 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb98b7-c6af-43ca-9b20-6f5568344478 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:41.674440 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb98b7-c6af-43ca-9b20-6f5568344478 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:36:41.674883 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb98b7-c6af-43ca-9b20-6f5568344478 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:41.676475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb98b7-c6af-43ca-9b20-6f5568344478 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:41.688724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1074/2111] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:36:41.693872 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a6e3b-3f36-4651-aa22-a9e03c9daa9d None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:41.716104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a6e3b-3f36-4651-aa22-a9e03c9daa9d None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:41.721083 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4361d-07a9-49d2-84d7-7e21ba8475a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:41.721793 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4361d-07a9-49d2-84d7-7e21ba8475a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:41.722075 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4361d-07a9-49d2-84d7-7e21ba8475a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:41.722355 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4361d-07a9-49d2-84d7-7e21ba8475a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:41.723057 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4361d-07a9-49d2-84d7-7e21ba8475a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:41.724995 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4361d-07a9-49d2-84d7-7e21ba8475a9 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:41.814169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb98b7-c6af-43ca-9b20-6f5568344478 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:41.816066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d2eb98b7-c6af-43ca-9b20-6f5568344478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:41.832227 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb98b7-c6af-43ca-9b20-6f5568344478 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:41.837158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a7fee-bc0e-46c2-965d-c44206a73955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:41.837882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a7fee-bc0e-46c2-965d-c44206a73955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:41.838234 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a7fee-bc0e-46c2-965d-c44206a73955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:41.838579 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a7fee-bc0e-46c2-965d-c44206a73955 admin admin] PATH_INFO: `/v3/users/56e3961aa49a430291f924cfd4c190db` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:41.839967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a7fee-bc0e-46c2-965d-c44206a73955 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56e3961aa49a430291f924cfd4c190db)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:41.842221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a7fee-bc0e-46c2-965d-c44206a73955 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:41.853649 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1038/2112] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:41 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:36:41.857756 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4361d-07a9-49d2-84d7-7e21ba8475a9 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:41.919867 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4361d-07a9-49d2-84d7-7e21ba8475a9 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:41.925212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b28cce-2cd2-4b0f-955b-41e193245ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:41.925817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b28cce-2cd2-4b0f-955b-41e193245ff3 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:41.926100 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b28cce-2cd2-4b0f-955b-41e193245ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:41.926405 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b28cce-2cd2-4b0f-955b-41e193245ff3 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:41.927072 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b28cce-2cd2-4b0f-955b-41e193245ff3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:41.928993 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b28cce-2cd2-4b0f-955b-41e193245ff3 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:41.951839 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:42.055973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1039/2113] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:41 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:36:42.060479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b28cce-2cd2-4b0f-955b-41e193245ff3 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:42.067736 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f66a7fee-bc0e-46c2-965d-c44206a73955 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56e3961aa49a430291f924cfd4c190db'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:42.083963 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b28cce-2cd2-4b0f-955b-41e193245ff3 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:42.110131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8f23f-24e1-41b0-8bc2-cb48242d3ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:42.110682 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8f23f-24e1-41b0-8bc2-cb48242d3ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:42.110987 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8f23f-24e1-41b0-8bc2-cb48242d3ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:42.111261 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8f23f-24e1-41b0-8bc2-cb48242d3ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:42.111913 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8f23f-24e1-41b0-8bc2-cb48242d3ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:42.113520 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8f23f-24e1-41b0-8bc2-cb48242d3ae8 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:42.127995 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1040/2114] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:42.131916 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8f23f-24e1-41b0-8bc2-cb48242d3ae8 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:42.141842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8f23f-24e1-41b0-8bc2-cb48242d3ae8 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:42.146603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679b72f-cea3-47dd-a620-7cae81489e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:42.147257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679b72f-cea3-47dd-a620-7cae81489e89 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:42.147824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679b72f-cea3-47dd-a620-7cae81489e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:42.147824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679b72f-cea3-47dd-a620-7cae81489e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:42.148436 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679b72f-cea3-47dd-a620-7cae81489e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:42.150026 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679b72f-cea3-47dd-a620-7cae81489e89 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:42.160827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1041/2115] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:42.164787 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679b72f-cea3-47dd-a620-7cae81489e89 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:42.175003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679b72f-cea3-47dd-a620-7cae81489e89 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:42.179888 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47329d50-4a7f-4816-a2b2-7af8f06da980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:42.182841 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47329d50-4a7f-4816-a2b2-7af8f06da980 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:42.182841 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47329d50-4a7f-4816-a2b2-7af8f06da980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:42.182841 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47329d50-4a7f-4816-a2b2-7af8f06da980 admin admin] PATH_INFO: `/v3/projects/3bd167e7a5a84dedb7d534ee8eefcb0f/users/7a489a31d97d46c4b8e1acbfd8200d79/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:42.196040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47329d50-4a7f-4816-a2b2-7af8f06da980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd167e7a5a84dedb7d534ee8eefcb0f, user_id=7a489a31d97d46c4b8e1acbfd8200d79, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:42.198939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47329d50-4a7f-4816-a2b2-7af8f06da980 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:42.200593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f66a7fee-bc0e-46c2-965d-c44206a73955 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56e3961aa49a430291f924cfd4c190db'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:42.214967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f66a7fee-bc0e-46c2-965d-c44206a73955 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56e3961aa49a430291f924cfd4c190db'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:42.216004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f66a7fee-bc0e-46c2-965d-c44206a73955 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56e3961aa49a430291f924cfd4c190db'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:42.223818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f66a7fee-bc0e-46c2-965d-c44206a73955 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56e3961aa49a430291f924cfd4c190db'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:42.229954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1075/2116] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:41 2022] DELETE /identity/v3/users/56e3961aa49a430291f924cfd4c190db => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:42.292975 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1042/2117] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:42 2022] PUT /identity/v3/projects/3bd167e7a5a84dedb7d534ee8eefcb0f/users/7a489a31d97d46c4b8e1acbfd8200d79/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:42.303346 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fc64e-beae-4c51-b958-511647c21e05 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:42.303638 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fc64e-beae-4c51-b958-511647c21e05 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:42.303973 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fc64e-beae-4c51-b958-511647c21e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:42.357908 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-d47fc64e-beae-4c51-b958-511647c21e05 None None] MFA Rules not processed for user `7a489a31d97d46c4b8e1acbfd8200d79`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:42.361024 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d47fc64e-beae-4c51-b958-511647c21e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:42.395130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:42.449186 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1076/2118] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:42.460695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47329d50-4a7f-4816-a2b2-7af8f06da980 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:42.462431 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-47329d50-4a7f-4816-a2b2-7af8f06da980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:42.522049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47329d50-4a7f-4816-a2b2-7af8f06da980 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:42.525902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fdc389-ba87-4bdc-ae48-6c40908fffe2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:42.526516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdc389-ba87-4bdc-ae48-6c40908fffe2 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:42.526809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdc389-ba87-4bdc-ae48-6c40908fffe2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:42.527102 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdc389-ba87-4bdc-ae48-6c40908fffe2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:42.540968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fc64e-beae-4c51-b958-511647c21e05 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:42.542731 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d47fc64e-beae-4c51-b958-511647c21e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:42.546581 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fdc389-ba87-4bdc-ae48-6c40908fffe2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:42.548812 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fdc389-ba87-4bdc-ae48-6c40908fffe2 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:42.558816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1043/2119] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:42.631370 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fc64e-beae-4c51-b958-511647c21e05 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:42.637436 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66eee6-0cfb-45ca-8e06-93b663184d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:42.638079 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66eee6-0cfb-45ca-8e06-93b663184d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:42.638389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66eee6-0cfb-45ca-8e06-93b663184d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:42.638668 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66eee6-0cfb-45ca-8e06-93b663184d1f admin admin] PATH_INFO: `/v3/projects/a6d83fb65c95456d995b530844e6797c` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:42.639827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66eee6-0cfb-45ca-8e06-93b663184d1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6d83fb65c95456d995b530844e6797c)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:42.642303 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66eee6-0cfb-45ca-8e06-93b663184d1f admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:42.777583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0e66eee6-0cfb-45ca-8e06-93b663184d1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6d83fb65c95456d995b530844e6797c', 'request_id': 'req-0e66eee6-0cfb-45ca-8e06-93b663184d1f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:42.783758 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0e66eee6-0cfb-45ca-8e06-93b663184d1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6d83fb65c95456d995b530844e6797c', 'request_id': 'req-0e66eee6-0cfb-45ca-8e06-93b663184d1f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:42.895984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0e66eee6-0cfb-45ca-8e06-93b663184d1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6d83fb65c95456d995b530844e6797c', 'request_id': 'req-0e66eee6-0cfb-45ca-8e06-93b663184d1f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:42.907419 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0e66eee6-0cfb-45ca-8e06-93b663184d1f admin admin] The token cache is being invalidate because project a6d83fb65c95456d995b530844e6797c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:42.907712 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0e66eee6-0cfb-45ca-8e06-93b663184d1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:42.908511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0e66eee6-0cfb-45ca-8e06-93b663184d1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:42.910233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1077/2120] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:42 2022] DELETE /identity/v3/projects/a6d83fb65c95456d995b530844e6797c => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:42.914362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fdc389-ba87-4bdc-ae48-6c40908fffe2 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:42.925607 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fdc389-ba87-4bdc-ae48-6c40908fffe2 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:42.985830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d084355-082c-4c80-92de-32cd77450b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:42.986580 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d084355-082c-4c80-92de-32cd77450b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:42.986944 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d084355-082c-4c80-92de-32cd77450b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:42.987289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d084355-082c-4c80-92de-32cd77450b1f admin admin] PATH_INFO: `/v3/users/3df1a36fe2c74e6c9b0e7449e48b0549` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:42.988544 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d084355-082c-4c80-92de-32cd77450b1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3df1a36fe2c74e6c9b0e7449e48b0549)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:42.990697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d084355-082c-4c80-92de-32cd77450b1f admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:43.376098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2d084355-082c-4c80-92de-32cd77450b1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3df1a36fe2c74e6c9b0e7449e48b0549'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:43.503716 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2d084355-082c-4c80-92de-32cd77450b1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3df1a36fe2c74e6c9b0e7449e48b0549'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:43.504442 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2d084355-082c-4c80-92de-32cd77450b1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3df1a36fe2c74e6c9b0e7449e48b0549'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:43.519045 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2d084355-082c-4c80-92de-32cd77450b1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3df1a36fe2c74e6c9b0e7449e48b0549'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:43.527787 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2d084355-082c-4c80-92de-32cd77450b1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3df1a36fe2c74e6c9b0e7449e48b0549'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:43.537309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1044/2121] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:42 2022] DELETE /identity/v3/users/3df1a36fe2c74e6c9b0e7449e48b0549 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:43.792587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66eee6-0cfb-45ca-8e06-93b663184d1f admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:43.796346 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0e66eee6-0cfb-45ca-8e06-93b663184d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:43.816310 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66eee6-0cfb-45ca-8e06-93b663184d1f admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:43.887528 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62730a-21c5-40a5-819b-c9986eb5cf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:43.888078 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62730a-21c5-40a5-819b-c9986eb5cf5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:43.888360 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62730a-21c5-40a5-819b-c9986eb5cf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:43.888646 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62730a-21c5-40a5-819b-c9986eb5cf5b admin admin] PATH_INFO: `/v3/projects/ec8ee3a8020e499189c10f89d19acafd` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:43.889746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62730a-21c5-40a5-819b-c9986eb5cf5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec8ee3a8020e499189c10f89d19acafd)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:43.892309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62730a-21c5-40a5-819b-c9986eb5cf5b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:44.046706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-4b62730a-21c5-40a5-819b-c9986eb5cf5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec8ee3a8020e499189c10f89d19acafd', 'request_id': 'req-4b62730a-21c5-40a5-819b-c9986eb5cf5b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:44.053441 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-4b62730a-21c5-40a5-819b-c9986eb5cf5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec8ee3a8020e499189c10f89d19acafd', 'request_id': 'req-4b62730a-21c5-40a5-819b-c9986eb5cf5b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:44.143402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-4b62730a-21c5-40a5-819b-c9986eb5cf5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec8ee3a8020e499189c10f89d19acafd', 'request_id': 'req-4b62730a-21c5-40a5-819b-c9986eb5cf5b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:44.154208 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-4b62730a-21c5-40a5-819b-c9986eb5cf5b admin admin] The token cache is being invalidate because project ec8ee3a8020e499189c10f89d19acafd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:44.154492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-4b62730a-21c5-40a5-819b-c9986eb5cf5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:44.155121 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-4b62730a-21c5-40a5-819b-c9986eb5cf5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:44.157145 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1078/2122] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:43 2022] DELETE /identity/v3/projects/ec8ee3a8020e499189c10f89d19acafd => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:44.162014 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9679b8d-4573-4aa9-b3f4-08f19392eac7 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:44.162506 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9679b8d-4573-4aa9-b3f4-08f19392eac7 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:44.162625 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9679b8d-4573-4aa9-b3f4-08f19392eac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:44.184903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-e9679b8d-4573-4aa9-b3f4-08f19392eac7 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:44.186184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e9679b8d-4573-4aa9-b3f4-08f19392eac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:44.259327 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1045/2123] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:44.267249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fd541-f08f-455b-b1fc-aba0098275ee None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:44.267606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fd541-f08f-455b-b1fc-aba0098275ee None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:44.267881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fd541-f08f-455b-b1fc-aba0098275ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:44.287716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-315fd541-f08f-455b-b1fc-aba0098275ee None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:44.288946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-315fd541-f08f-455b-b1fc-aba0098275ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:44.297780 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1079/2124] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:44.302127 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9679b8d-4573-4aa9-b3f4-08f19392eac7 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:44.313165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9679b8d-4573-4aa9-b3f4-08f19392eac7 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:44.317940 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf5302-dd20-4495-85f2-8f3f0125e44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:44.318551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf5302-dd20-4495-85f2-8f3f0125e44c admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:44.318869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf5302-dd20-4495-85f2-8f3f0125e44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:44.319151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf5302-dd20-4495-85f2-8f3f0125e44c admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:44.319806 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf5302-dd20-4495-85f2-8f3f0125e44c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:36:44.320107 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf5302-dd20-4495-85f2-8f3f0125e44c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:44.321706 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf5302-dd20-4495-85f2-8f3f0125e44c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:44.334221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1046/2125] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:36:44.338643 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fd541-f08f-455b-b1fc-aba0098275ee None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:44.348976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fd541-f08f-455b-b1fc-aba0098275ee None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:44.353705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dbffe-71d0-4b55-a105-a43170f41f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:44.354296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dbffe-71d0-4b55-a105-a43170f41f43 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:44.354577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dbffe-71d0-4b55-a105-a43170f41f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:44.354884 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dbffe-71d0-4b55-a105-a43170f41f43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:44.355526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7dbffe-71d0-4b55-a105-a43170f41f43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:44.357509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7dbffe-71d0-4b55-a105-a43170f41f43 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:44.382471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:44.436398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1080/2126] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:44 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:44.440650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf5302-dd20-4495-85f2-8f3f0125e44c admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:44.450650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf5302-dd20-4495-85f2-8f3f0125e44c admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:44.507899 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dad9d-1ad0-4ea5-a933-63bbc629a287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:44.508514 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dad9d-1ad0-4ea5-a933-63bbc629a287 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:44.509243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dad9d-1ad0-4ea5-a933-63bbc629a287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:44.509243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dad9d-1ad0-4ea5-a933-63bbc629a287 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:44.509851 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dad9d-1ad0-4ea5-a933-63bbc629a287 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:44.511806 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dad9d-1ad0-4ea5-a933-63bbc629a287 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:44.537996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:44.641563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1047/2127] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:44 2022] POST /identity/v3/users => generated 285 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:44.645275 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dbffe-71d0-4b55-a105-a43170f41f43 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:44.655897 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dbffe-71d0-4b55-a105-a43170f41f43 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:44.661086 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbcefb-7761-485b-ad77-8be6da6b0642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:44.661680 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbcefb-7761-485b-ad77-8be6da6b0642 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:44.661958 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbcefb-7761-485b-ad77-8be6da6b0642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:44.662230 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbcefb-7761-485b-ad77-8be6da6b0642 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:44.662928 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbcefb-7761-485b-ad77-8be6da6b0642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:44.664554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbcefb-7761-485b-ad77-8be6da6b0642 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:44.674350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1081/2128] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:44.678321 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dad9d-1ad0-4ea5-a933-63bbc629a287 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:44.688617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dad9d-1ad0-4ea5-a933-63bbc629a287 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:44.693394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c45bd-fdfc-4aad-98a8-64769598f6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:44.694027 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c45bd-fdfc-4aad-98a8-64769598f6ec admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:44.694340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c45bd-fdfc-4aad-98a8-64769598f6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:44.694621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c45bd-fdfc-4aad-98a8-64769598f6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:44.695313 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c45bd-fdfc-4aad-98a8-64769598f6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:44.696993 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c45bd-fdfc-4aad-98a8-64769598f6ec admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:44.708275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1048/2129] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:44.712244 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbcefb-7761-485b-ad77-8be6da6b0642 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:44.722881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbcefb-7761-485b-ad77-8be6da6b0642 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:44.727568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078b0c0-55e9-4b08-bf6f-cb9b38579782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:44.728205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078b0c0-55e9-4b08-bf6f-cb9b38579782 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:44.728483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078b0c0-55e9-4b08-bf6f-cb9b38579782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:44.728776 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078b0c0-55e9-4b08-bf6f-cb9b38579782 admin admin] PATH_INFO: `/v3/projects/07cfda046f564ce881b1f0590beb90ef/users/d361b73dbc614e5cb7251fbe9e20fc02/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:44.744987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078b0c0-55e9-4b08-bf6f-cb9b38579782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07cfda046f564ce881b1f0590beb90ef, user_id=d361b73dbc614e5cb7251fbe9e20fc02, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:44.747885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078b0c0-55e9-4b08-bf6f-cb9b38579782 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:44.857791 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1082/2130] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:44 2022] PUT /identity/v3/projects/07cfda046f564ce881b1f0590beb90ef/users/d361b73dbc614e5cb7251fbe9e20fc02/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:44.869152 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e66308-027f-4125-9bf6-ab2816c52f5b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:44.869440 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e66308-027f-4125-9bf6-ab2816c52f5b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:44.869718 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e66308-027f-4125-9bf6-ab2816c52f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:44.897616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-51e66308-027f-4125-9bf6-ab2816c52f5b None None] MFA Rules not processed for user `d361b73dbc614e5cb7251fbe9e20fc02`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:44.898837 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-51e66308-027f-4125-9bf6-ab2816c52f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:44.966618 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1049/2131] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:44.971171 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078b0c0-55e9-4b08-bf6f-cb9b38579782 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:45.045658 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078b0c0-55e9-4b08-bf6f-cb9b38579782 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:45.050577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e14e0c-477b-4c57-9479-5e1f9c8cb2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:45.051272 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e14e0c-477b-4c57-9479-5e1f9c8cb2e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:45.051618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e14e0c-477b-4c57-9479-5e1f9c8cb2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:45.051890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e14e0c-477b-4c57-9479-5e1f9c8cb2e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:45.052547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e14e0c-477b-4c57-9479-5e1f9c8cb2e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:45.054736 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e14e0c-477b-4c57-9479-5e1f9c8cb2e1 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:45.489584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1083/2132] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:44 2022] POST /identity/v3/projects => generated 367 bytes in 519 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:45.493612 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e66308-027f-4125-9bf6-ab2816c52f5b None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:45.504949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e66308-027f-4125-9bf6-ab2816c52f5b None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:45.560276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54168cf4-fffb-42f8-84bd-fdfe14bf6967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:45.560809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54168cf4-fffb-42f8-84bd-fdfe14bf6967 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:45.561092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54168cf4-fffb-42f8-84bd-fdfe14bf6967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:45.561367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54168cf4-fffb-42f8-84bd-fdfe14bf6967 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:45.562027 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54168cf4-fffb-42f8-84bd-fdfe14bf6967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:45.564669 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54168cf4-fffb-42f8-84bd-fdfe14bf6967 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:45.819541 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1050/2133] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:45 2022] POST /identity/v3/users => generated 290 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:45.823672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e14e0c-477b-4c57-9479-5e1f9c8cb2e1 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:45.834030 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e14e0c-477b-4c57-9479-5e1f9c8cb2e1 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:45.839189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f15426e-1e64-4d1e-80ae-64007d9ee760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:45.839957 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15426e-1e64-4d1e-80ae-64007d9ee760 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:45.840236 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15426e-1e64-4d1e-80ae-64007d9ee760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:45.840766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15426e-1e64-4d1e-80ae-64007d9ee760 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:45.841405 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f15426e-1e64-4d1e-80ae-64007d9ee760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:45.843230 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f15426e-1e64-4d1e-80ae-64007d9ee760 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:45.855315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1084/2134] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:45.860196 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54168cf4-fffb-42f8-84bd-fdfe14bf6967 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:45.876166 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54168cf4-fffb-42f8-84bd-fdfe14bf6967 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:45.881237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68162dc5-8c4e-4f40-b6d9-1737b2879985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:45.881899 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68162dc5-8c4e-4f40-b6d9-1737b2879985 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:45.882185 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68162dc5-8c4e-4f40-b6d9-1737b2879985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:45.882461 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68162dc5-8c4e-4f40-b6d9-1737b2879985 admin admin] PATH_INFO: `/v3/projects/809e7784788a4d249d2f467b7bdadf3a/users/89b52086d30c4305a3d506d128df86a8/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:45.897442 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68162dc5-8c4e-4f40-b6d9-1737b2879985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=809e7784788a4d249d2f467b7bdadf3a, user_id=89b52086d30c4305a3d506d128df86a8, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:45.900250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68162dc5-8c4e-4f40-b6d9-1737b2879985 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:46.029969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1051/2135] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:45 2022] PUT /identity/v3/projects/809e7784788a4d249d2f467b7bdadf3a/users/89b52086d30c4305a3d506d128df86a8/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:46.034862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f15426e-1e64-4d1e-80ae-64007d9ee760 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:46.044894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f15426e-1e64-4d1e-80ae-64007d9ee760 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:46.103755 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0550e-f09f-4611-8a96-8928646f61fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:46.104427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0550e-f09f-4611-8a96-8928646f61fc admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:46.104728 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0550e-f09f-4611-8a96-8928646f61fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:46.105006 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0550e-f09f-4611-8a96-8928646f61fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:46.105669 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0550e-f09f-4611-8a96-8928646f61fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:46.107432 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0550e-f09f-4611-8a96-8928646f61fc admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:46.118236 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1085/2136] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:46.128333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68162dc5-8c4e-4f40-b6d9-1737b2879985 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:46.133001 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68162dc5-8c4e-4f40-b6d9-1737b2879985 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:46.137720 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65f751-51ad-4d5d-b42e-dff23449c455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:46.138365 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65f751-51ad-4d5d-b42e-dff23449c455 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:46.138672 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65f751-51ad-4d5d-b42e-dff23449c455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:46.138982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65f751-51ad-4d5d-b42e-dff23449c455 admin admin] PATH_INFO: `/v3/projects/809e7784788a4d249d2f467b7bdadf3a/users/89b52086d30c4305a3d506d128df86a8/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:46.141496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c65f751-51ad-4d5d-b42e-dff23449c455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=809e7784788a4d249d2f467b7bdadf3a, user_id=89b52086d30c4305a3d506d128df86a8, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:46.144160 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c65f751-51ad-4d5d-b42e-dff23449c455 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:46.157809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1052/2137] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:46 2022] PUT /identity/v3/projects/809e7784788a4d249d2f467b7bdadf3a/users/89b52086d30c4305a3d506d128df86a8/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:46.168085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3d541-3cf8-462d-a6fb-ae1f3f7f91fc None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:46.168371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3d541-3cf8-462d-a6fb-ae1f3f7f91fc None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:46.168648 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3d541-3cf8-462d-a6fb-ae1f3f7f91fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:46.207210 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-91d3d541-3cf8-462d-a6fb-ae1f3f7f91fc None None] MFA Rules not processed for user `89b52086d30c4305a3d506d128df86a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:46.208521 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-91d3d541-3cf8-462d-a6fb-ae1f3f7f91fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:46.296324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1086/2138] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:46.312597 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65f751-51ad-4d5d-b42e-dff23449c455 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:46.314721 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-5c65f751-51ad-4d5d-b42e-dff23449c455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:46.396013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65f751-51ad-4d5d-b42e-dff23449c455 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:46.399911 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752ec42-1f6a-4d6c-a8bc-e726b301a1fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:46.400521 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752ec42-1f6a-4d6c-a8bc-e726b301a1fb service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:46.400868 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752ec42-1f6a-4d6c-a8bc-e726b301a1fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:46.401069 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752ec42-1f6a-4d6c-a8bc-e726b301a1fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:46.423627 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752ec42-1f6a-4d6c-a8bc-e726b301a1fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:46.425097 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752ec42-1f6a-4d6c-a8bc-e726b301a1fb service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:46.436312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1053/2139] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:46.570905 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d3d541-3cf8-462d-a6fb-ae1f3f7f91fc None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:46.581435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d3d541-3cf8-462d-a6fb-ae1f3f7f91fc None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:46.585439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35a483-d08e-4b4d-82f1-353a2776c880 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:46.586131 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35a483-d08e-4b4d-82f1-353a2776c880 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:46.586511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35a483-d08e-4b4d-82f1-353a2776c880 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:46.586893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35a483-d08e-4b4d-82f1-353a2776c880 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:46.646676 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f35a483-d08e-4b4d-82f1-353a2776c880 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:46.648283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f35a483-d08e-4b4d-82f1-353a2776c880 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:46.658278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1087/2140] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:46.727054 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:47.001532 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:47.989541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:48.217799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:48.405015 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752ec42-1f6a-4d6c-a8bc-e726b301a1fb service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:48.406484 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-6752ec42-1f6a-4d6c-a8bc-e726b301a1fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:48.471612 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752ec42-1f6a-4d6c-a8bc-e726b301a1fb service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:48.476166 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e9ae7-8b4c-4733-8150-1346cc31265b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:48.476796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e9ae7-8b4c-4733-8150-1346cc31265b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:48.477078 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e9ae7-8b4c-4733-8150-1346cc31265b admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:48.477355 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e9ae7-8b4c-4733-8150-1346cc31265b admin admin] PATH_INFO: `/v3/users/49192fcffa9e41b5971cdb1b337f8c9a` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:48.478458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e9ae7-8b4c-4733-8150-1346cc31265b admin admin] RBAC: Authorizing `identity:delete_user(user_id=49192fcffa9e41b5971cdb1b337f8c9a)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:48.480413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e9ae7-8b4c-4733-8150-1346cc31265b admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:48.684963 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f15e9ae7-8b4c-4733-8150-1346cc31265b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49192fcffa9e41b5971cdb1b337f8c9a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:48.788938 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f15e9ae7-8b4c-4733-8150-1346cc31265b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49192fcffa9e41b5971cdb1b337f8c9a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:48.789559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f15e9ae7-8b4c-4733-8150-1346cc31265b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49192fcffa9e41b5971cdb1b337f8c9a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:48.803161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f15e9ae7-8b4c-4733-8150-1346cc31265b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49192fcffa9e41b5971cdb1b337f8c9a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:48.809994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f15e9ae7-8b4c-4733-8150-1346cc31265b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49192fcffa9e41b5971cdb1b337f8c9a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:48.816241 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1054/2141] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:48 2022] DELETE /identity/v3/users/49192fcffa9e41b5971cdb1b337f8c9a => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:48.820990 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610598a-8b25-4bf7-a7ac-c1025399666d None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:48.821304 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610598a-8b25-4bf7-a7ac-c1025399666d None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:48.821624 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610598a-8b25-4bf7-a7ac-c1025399666d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:48.841055 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-b610598a-8b25-4bf7-a7ac-c1025399666d None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:48.842334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b610598a-8b25-4bf7-a7ac-c1025399666d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:48.904935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1088/2142] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:48.915350 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e9ae7-8b4c-4733-8150-1346cc31265b admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:48.916753 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f15e9ae7-8b4c-4733-8150-1346cc31265b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:48.968039 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d4e90-77a4-4302-b27b-3814896c6af0 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:48.968326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d4e90-77a4-4302-b27b-3814896c6af0 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:48.968628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d4e90-77a4-4302-b27b-3814896c6af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:48.973901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e9ae7-8b4c-4733-8150-1346cc31265b admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:48.977802 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76d464-cf82-4cf2-b001-5f1f3e029694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:48.978435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76d464-cf82-4cf2-b001-5f1f3e029694 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:48.978720 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76d464-cf82-4cf2-b001-5f1f3e029694 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:48.979049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76d464-cf82-4cf2-b001-5f1f3e029694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:48.987975 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-cd8d4e90-77a4-4302-b27b-3814896c6af0 None None] MFA Rules not processed for user `9136f02e6c664f27b3f4fd35475f0eea`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:48.997676 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-cd8d4e90-77a4-4302-b27b-3814896c6af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:48.998442 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76d464-cf82-4cf2-b001-5f1f3e029694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:48.998442 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76d464-cf82-4cf2-b001-5f1f3e029694 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:49.003015 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1055/2143] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:49.048774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1089/2144] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:49.622536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76d464-cf82-4cf2-b001-5f1f3e029694 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:49.622536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-2f76d464-cf82-4cf2-b001-5f1f3e029694 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:49.637118 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76d464-cf82-4cf2-b001-5f1f3e029694 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:49.642208 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:49.642974 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:49.643342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:49.644048 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41 admin admin] PATH_INFO: `/v3/projects/eea97abb092249868b4aaef4d8f0c4c3` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:49.646232 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eea97abb092249868b4aaef4d8f0c4c3)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:49.653859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:49.830440 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eea97abb092249868b4aaef4d8f0c4c3', 'request_id': 'req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:49.836318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eea97abb092249868b4aaef4d8f0c4c3', 'request_id': 'req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:49.955459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:49.989434 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eea97abb092249868b4aaef4d8f0c4c3', 'request_id': 'req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:50.002499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41 admin admin] The token cache is being invalidate because project eea97abb092249868b4aaef4d8f0c4c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:50.002806 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:50.003500 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:50.005064 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1056/2145] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:49 2022] DELETE /identity/v3/projects/eea97abb092249868b4aaef4d8f0c4c3 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:50.009246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d4e90-77a4-4302-b27b-3814896c6af0 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:50.020610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d4e90-77a4-4302-b27b-3814896c6af0 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:50.079488 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd4c65-7a21-4114-9354-12b43cbe4dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:50.080146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd4c65-7a21-4114-9354-12b43cbe4dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:50.080469 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd4c65-7a21-4114-9354-12b43cbe4dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:50.080747 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd4c65-7a21-4114-9354-12b43cbe4dbc admin admin] PATH_INFO: `/v3/users/5baea20e78174f92a0c6bfb3ab14741f` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:50.081882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd4c65-7a21-4114-9354-12b43cbe4dbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5baea20e78174f92a0c6bfb3ab14741f)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:50.083920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd4c65-7a21-4114-9354-12b43cbe4dbc admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:50.149078 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:50.368123 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-19cd4c65-7a21-4114-9354-12b43cbe4dbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5baea20e78174f92a0c6bfb3ab14741f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:50.463275 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-19cd4c65-7a21-4114-9354-12b43cbe4dbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5baea20e78174f92a0c6bfb3ab14741f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:50.475728 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-19cd4c65-7a21-4114-9354-12b43cbe4dbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5baea20e78174f92a0c6bfb3ab14741f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:50.476354 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-19cd4c65-7a21-4114-9354-12b43cbe4dbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5baea20e78174f92a0c6bfb3ab14741f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:50.483394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-19cd4c65-7a21-4114-9354-12b43cbe4dbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5baea20e78174f92a0c6bfb3ab14741f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:50.489262 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1090/2146] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:50 2022] DELETE /identity/v3/users/5baea20e78174f92a0c6bfb3ab14741f => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:50.971081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:50.973017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:50.989412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bac0c0-3bb6-408b-a32b-3fb448c49f41 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:51.048083 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9843b70-834c-4d55-acee-f48ce1f20b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:51.048723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9843b70-834c-4d55-acee-f48ce1f20b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:51.049004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9843b70-834c-4d55-acee-f48ce1f20b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:51.049275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9843b70-834c-4d55-acee-f48ce1f20b67 admin admin] PATH_INFO: `/v3/projects/57bf2f52c9214b9dbd69af9981905a95` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:51.050639 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9843b70-834c-4d55-acee-f48ce1f20b67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57bf2f52c9214b9dbd69af9981905a95)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:51.052817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9843b70-834c-4d55-acee-f48ce1f20b67 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:51.241538 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a9843b70-834c-4d55-acee-f48ce1f20b67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57bf2f52c9214b9dbd69af9981905a95', 'request_id': 'req-a9843b70-834c-4d55-acee-f48ce1f20b67'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:51.248511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a9843b70-834c-4d55-acee-f48ce1f20b67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57bf2f52c9214b9dbd69af9981905a95', 'request_id': 'req-a9843b70-834c-4d55-acee-f48ce1f20b67'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:51.334503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a9843b70-834c-4d55-acee-f48ce1f20b67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57bf2f52c9214b9dbd69af9981905a95', 'request_id': 'req-a9843b70-834c-4d55-acee-f48ce1f20b67'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:51.351629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a9843b70-834c-4d55-acee-f48ce1f20b67 admin admin] The token cache is being invalidate because project 57bf2f52c9214b9dbd69af9981905a95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:51.351831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a9843b70-834c-4d55-acee-f48ce1f20b67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:51.354158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-a9843b70-834c-4d55-acee-f48ce1f20b67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:51.359076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1057/2147] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:50 2022] DELETE /identity/v3/projects/57bf2f52c9214b9dbd69af9981905a95 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:51.361781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917aab7e-87eb-4297-bf3c-a54021e8f87d None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:51.362252 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917aab7e-87eb-4297-bf3c-a54021e8f87d None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:51.362374 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917aab7e-87eb-4297-bf3c-a54021e8f87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:51.392924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-917aab7e-87eb-4297-bf3c-a54021e8f87d None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:51.394228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-917aab7e-87eb-4297-bf3c-a54021e8f87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:51.470343 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1091/2148] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:51.478691 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8b40f-2d3a-48dd-971a-1ed818f35205 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:51.479014 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8b40f-2d3a-48dd-971a-1ed818f35205 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:51.479381 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8b40f-2d3a-48dd-971a-1ed818f35205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:51.506995 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-6be8b40f-2d3a-48dd-971a-1ed818f35205 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:51.508322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-6be8b40f-2d3a-48dd-971a-1ed818f35205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:51.526228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917aab7e-87eb-4297-bf3c-a54021e8f87d None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:51.530468 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1058/2149] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:51.543183 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917aab7e-87eb-4297-bf3c-a54021e8f87d None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:51.552028 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d6895-ddcc-47d2-8452-b495bc8c5400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:51.552858 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d6895-ddcc-47d2-8452-b495bc8c5400 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:51.553291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d6895-ddcc-47d2-8452-b495bc8c5400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:51.553791 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d6895-ddcc-47d2-8452-b495bc8c5400 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:51.554616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d6895-ddcc-47d2-8452-b495bc8c5400 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:36:51.554961 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d6895-ddcc-47d2-8452-b495bc8c5400 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:51.557522 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d6895-ddcc-47d2-8452-b495bc8c5400 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:51.575243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1092/2150] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:36:51.577034 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8b40f-2d3a-48dd-971a-1ed818f35205 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:51.593143 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8b40f-2d3a-48dd-971a-1ed818f35205 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:51.615751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e061a8e-15b8-4c19-9041-290ccebec3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:51.617461 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e061a8e-15b8-4c19-9041-290ccebec3a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:51.618731 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e061a8e-15b8-4c19-9041-290ccebec3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:51.622278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e061a8e-15b8-4c19-9041-290ccebec3a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:51.623156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e061a8e-15b8-4c19-9041-290ccebec3a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:51.625399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e061a8e-15b8-4c19-9041-290ccebec3a7 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:51.795491 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1059/2151] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:51 2022] POST /identity/v3/projects => generated 379 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:51.796742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d6895-ddcc-47d2-8452-b495bc8c5400 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:51.815231 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d6895-ddcc-47d2-8452-b495bc8c5400 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:51.915418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc65d9-1441-4b58-b647-28f32cac619b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:51.916043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc65d9-1441-4b58-b647-28f32cac619b admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:51.918951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc65d9-1441-4b58-b647-28f32cac619b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:51.918951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc65d9-1441-4b58-b647-28f32cac619b admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:51.918951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc65d9-1441-4b58-b647-28f32cac619b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:51.919284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc65d9-1441-4b58-b647-28f32cac619b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:52.059563 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1093/2152] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:51 2022] POST /identity/v3/users => generated 290 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:52.064362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e061a8e-15b8-4c19-9041-290ccebec3a7 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:52.076430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e061a8e-15b8-4c19-9041-290ccebec3a7 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:52.081827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951d6fa-ecb7-44c6-ab78-d1e6a2708b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:52.091297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951d6fa-ecb7-44c6-ab78-d1e6a2708b52 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:52.091297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951d6fa-ecb7-44c6-ab78-d1e6a2708b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:52.091297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951d6fa-ecb7-44c6-ab78-d1e6a2708b52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:52.091297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951d6fa-ecb7-44c6-ab78-d1e6a2708b52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:52.091297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951d6fa-ecb7-44c6-ab78-d1e6a2708b52 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:52.115206 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1060/2153] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:52.122167 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc65d9-1441-4b58-b647-28f32cac619b admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:52.137010 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc65d9-1441-4b58-b647-28f32cac619b admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:52.142745 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26409d-232e-4a01-8ff8-21ae2406ce24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:52.143470 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26409d-232e-4a01-8ff8-21ae2406ce24 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:52.143751 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26409d-232e-4a01-8ff8-21ae2406ce24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:52.144028 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26409d-232e-4a01-8ff8-21ae2406ce24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:52.144713 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26409d-232e-4a01-8ff8-21ae2406ce24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:52.146314 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26409d-232e-4a01-8ff8-21ae2406ce24 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:52.159764 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1094/2154] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:52.164123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951d6fa-ecb7-44c6-ab78-d1e6a2708b52 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:52.189673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951d6fa-ecb7-44c6-ab78-d1e6a2708b52 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:52.195183 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b96d7-5eb2-4590-a89f-855745c1bcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:52.195850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b96d7-5eb2-4590-a89f-855745c1bcff admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:52.196133 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b96d7-5eb2-4590-a89f-855745c1bcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:52.196409 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b96d7-5eb2-4590-a89f-855745c1bcff admin admin] PATH_INFO: `/v3/projects/e3c85a3787784f90a0814aed1762840f/users/ed240a780e2942eaafbc7b1aa33b1d0c/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:52.220372 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b96d7-5eb2-4590-a89f-855745c1bcff admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3c85a3787784f90a0814aed1762840f, user_id=ed240a780e2942eaafbc7b1aa33b1d0c, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:52.223444 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b96d7-5eb2-4590-a89f-855745c1bcff admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:52.315261 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1061/2155] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:52 2022] PUT /identity/v3/projects/e3c85a3787784f90a0814aed1762840f/users/ed240a780e2942eaafbc7b1aa33b1d0c/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:52.331890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba428c0-c0a7-454d-acda-147c2c977537 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:52.332175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba428c0-c0a7-454d-acda-147c2c977537 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:52.332449 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba428c0-c0a7-454d-acda-147c2c977537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:52.361977 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-eba428c0-c0a7-454d-acda-147c2c977537 None None] MFA Rules not processed for user `ed240a780e2942eaafbc7b1aa33b1d0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:52.363263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-eba428c0-c0a7-454d-acda-147c2c977537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:52.426255 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1095/2156] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:52.430677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b96d7-5eb2-4590-a89f-855745c1bcff admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:52.460319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba428c0-c0a7-454d-acda-147c2c977537 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:52.461723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-eba428c0-c0a7-454d-acda-147c2c977537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:52.497171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b96d7-5eb2-4590-a89f-855745c1bcff admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:52.502427 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679e068-8230-4a30-8866-578e5afc2901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:52.503215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679e068-8230-4a30-8866-578e5afc2901 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:52.503495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679e068-8230-4a30-8866-578e5afc2901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:52.503769 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679e068-8230-4a30-8866-578e5afc2901 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:52.504404 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679e068-8230-4a30-8866-578e5afc2901 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:52.506372 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679e068-8230-4a30-8866-578e5afc2901 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:52.539617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba428c0-c0a7-454d-acda-147c2c977537 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:52.544163 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0dea21-61da-4881-9850-ad6c7749f472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:52.544882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dea21-61da-4881-9850-ad6c7749f472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:52.545230 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dea21-61da-4881-9850-ad6c7749f472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:52.545721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dea21-61da-4881-9850-ad6c7749f472 admin admin] PATH_INFO: `/v3/users/d361b73dbc614e5cb7251fbe9e20fc02` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:52.546986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0dea21-61da-4881-9850-ad6c7749f472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d361b73dbc614e5cb7251fbe9e20fc02)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:52.549212 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0dea21-61da-4881-9850-ad6c7749f472 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:52.659726 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1062/2157] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:52 2022] POST /identity/v3/projects => generated 379 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:52.663636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679e068-8230-4a30-8866-578e5afc2901 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:52.729850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679e068-8230-4a30-8866-578e5afc2901 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:52.734585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a6e00-dbdf-420a-93ea-ff2f3fe8fa9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:52.735278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a6e00-dbdf-420a-93ea-ff2f3fe8fa9d admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:52.735563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a6e00-dbdf-420a-93ea-ff2f3fe8fa9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:52.735839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a6e00-dbdf-420a-93ea-ff2f3fe8fa9d admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:52.736472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a6e00-dbdf-420a-93ea-ff2f3fe8fa9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:52.738381 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a6e00-dbdf-420a-93ea-ff2f3fe8fa9d admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:52.908717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1063/2158] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:52 2022] POST /identity/v3/users => generated 290 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:36:52.912398 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a6e00-dbdf-420a-93ea-ff2f3fe8fa9d admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:52.915526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-5c0dea21-61da-4881-9850-ad6c7749f472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd361b73dbc614e5cb7251fbe9e20fc02'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:52.922647 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a6e00-dbdf-420a-93ea-ff2f3fe8fa9d admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:52.927594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6005cd3-f33e-4a0a-8fad-f6c6e98b1b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:52.928800 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6005cd3-f33e-4a0a-8fad-f6c6e98b1b41 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:52.928800 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6005cd3-f33e-4a0a-8fad-f6c6e98b1b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:52.928999 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6005cd3-f33e-4a0a-8fad-f6c6e98b1b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:52.930018 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6005cd3-f33e-4a0a-8fad-f6c6e98b1b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:52.932089 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6005cd3-f33e-4a0a-8fad-f6c6e98b1b41 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:52.945929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1064/2159] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:52.949811 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6005cd3-f33e-4a0a-8fad-f6c6e98b1b41 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:52.959277 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6005cd3-f33e-4a0a-8fad-f6c6e98b1b41 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:52.964509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae909f5-6536-40d8-9b40-808593aabfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:52.965103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae909f5-6536-40d8-9b40-808593aabfb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:52.969489 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae909f5-6536-40d8-9b40-808593aabfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:52.969489 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae909f5-6536-40d8-9b40-808593aabfb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:52.969489 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae909f5-6536-40d8-9b40-808593aabfb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:52.969489 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae909f5-6536-40d8-9b40-808593aabfb6 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:52.978691 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1065/2160] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:52.982552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae909f5-6536-40d8-9b40-808593aabfb6 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:52.990243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-5c0dea21-61da-4881-9850-ad6c7749f472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd361b73dbc614e5cb7251fbe9e20fc02'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:52.998582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:52.998719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae909f5-6536-40d8-9b40-808593aabfb6 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:53.002822 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e225e-421c-46d9-bbe5-8aa6954092c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:53.003143 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e225e-421c-46d9-bbe5-8aa6954092c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:53.003421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e225e-421c-46d9-bbe5-8aa6954092c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:53.003692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e225e-421c-46d9-bbe5-8aa6954092c8 admin admin] PATH_INFO: `/v3/projects/b9a8fc9c0d8d47ef9afe102f352f48e6/users/44bdf7e64f034961addb9f1a34ae0a34/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:53.007683 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-5c0dea21-61da-4881-9850-ad6c7749f472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd361b73dbc614e5cb7251fbe9e20fc02'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:53.008486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-5c0dea21-61da-4881-9850-ad6c7749f472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd361b73dbc614e5cb7251fbe9e20fc02'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:53.015826 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-5c0dea21-61da-4881-9850-ad6c7749f472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd361b73dbc614e5cb7251fbe9e20fc02'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:53.020632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e225e-421c-46d9-bbe5-8aa6954092c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a8fc9c0d8d47ef9afe102f352f48e6, user_id=44bdf7e64f034961addb9f1a34ae0a34, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:53.026601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e225e-421c-46d9-bbe5-8aa6954092c8 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:53.026734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1096/2161] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:52 2022] DELETE /identity/v3/users/d361b73dbc614e5cb7251fbe9e20fc02 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:53.028440 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cd810-d1c1-4911-9a37-d6cac0a7af7b None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:53.028729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cd810-d1c1-4911-9a37-d6cac0a7af7b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:53.029007 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cd810-d1c1-4911-9a37-d6cac0a7af7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:53.048365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-225cd810-d1c1-4911-9a37-d6cac0a7af7b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:53.049610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-225cd810-d1c1-4911-9a37-d6cac0a7af7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:53.117872 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1066/2162] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:52 2022] PUT /identity/v3/projects/b9a8fc9c0d8d47ef9afe102f352f48e6/users/44bdf7e64f034961addb9f1a34ae0a34/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:53.121064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1097/2163] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:53.132714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e225e-421c-46d9-bbe5-8aa6954092c8 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:53.133152 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9b797-13d8-4492-a3fb-0b46ec5784b6 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:53.133504 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9b797-13d8-4492-a3fb-0b46ec5784b6 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:53.133849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9b797-13d8-4492-a3fb-0b46ec5784b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:53.135257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-5d7e225e-421c-46d9-bbe5-8aa6954092c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:53.165626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-5ea9b797-13d8-4492-a3fb-0b46ec5784b6 None None] MFA Rules not processed for user `44bdf7e64f034961addb9f1a34ae0a34`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:53.167183 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5ea9b797-13d8-4492-a3fb-0b46ec5784b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:53.197832 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e225e-421c-46d9-bbe5-8aa6954092c8 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:53.201687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060b8fd-06f1-4274-8970-85d3be6eadad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:53.202340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060b8fd-06f1-4274-8970-85d3be6eadad service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:53.202652 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060b8fd-06f1-4274-8970-85d3be6eadad service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:53.202968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060b8fd-06f1-4274-8970-85d3be6eadad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:53.243066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1098/2164] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:53.244381 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9b797-13d8-4492-a3fb-0b46ec5784b6 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:53.246016 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5ea9b797-13d8-4492-a3fb-0b46ec5784b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:53.297798 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:53.305221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060b8fd-06f1-4274-8970-85d3be6eadad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:53.306833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060b8fd-06f1-4274-8970-85d3be6eadad service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:53.332808 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1067/2165] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:53.344863 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9b797-13d8-4492-a3fb-0b46ec5784b6 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:53.351999 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4649148-4969-4052-a265-5f35612f8d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:53.352755 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4649148-4969-4052-a265-5f35612f8d02 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:53.353148 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4649148-4969-4052-a265-5f35612f8d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:53.353515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4649148-4969-4052-a265-5f35612f8d02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:53.354390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4649148-4969-4052-a265-5f35612f8d02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:53.357467 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4649148-4969-4052-a265-5f35612f8d02 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:53.475093 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1099/2166] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:53 2022] POST /identity/v3/projects => generated 379 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:36:53.476228 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060b8fd-06f1-4274-8970-85d3be6eadad service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:53.486713 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060b8fd-06f1-4274-8970-85d3be6eadad service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:53.584797 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a155b6f-b095-444a-9763-cd3ce1ea9670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:53.585414 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a155b6f-b095-444a-9763-cd3ce1ea9670 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:53.585695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a155b6f-b095-444a-9763-cd3ce1ea9670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:53.585968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a155b6f-b095-444a-9763-cd3ce1ea9670 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:53.586629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a155b6f-b095-444a-9763-cd3ce1ea9670 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:53.588570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a155b6f-b095-444a-9763-cd3ce1ea9670 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:53.687356 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1068/2167] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:53 2022] POST /identity/v3/users => generated 296 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:36:53.691873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4649148-4969-4052-a265-5f35612f8d02 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:53.703586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4649148-4969-4052-a265-5f35612f8d02 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:53.708454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a1573-189b-424c-bdc1-09464d3a695a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:53.709060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a1573-189b-424c-bdc1-09464d3a695a admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:53.709339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a1573-189b-424c-bdc1-09464d3a695a admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:53.709611 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a1573-189b-424c-bdc1-09464d3a695a admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:53.710247 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a1573-189b-424c-bdc1-09464d3a695a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:53.711900 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a1573-189b-424c-bdc1-09464d3a695a admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:53.724369 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1100/2168] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:53.728808 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a155b6f-b095-444a-9763-cd3ce1ea9670 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:53.739169 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a155b6f-b095-444a-9763-cd3ce1ea9670 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:53.744058 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52677ee-75d8-4527-88c2-c549b59118d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:53.744783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52677ee-75d8-4527-88c2-c549b59118d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:53.745074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52677ee-75d8-4527-88c2-c549b59118d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:53.745346 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52677ee-75d8-4527-88c2-c549b59118d6 admin admin] PATH_INFO: `/v3/projects/e5dab1c4dd4b4973ac2630ba9519f983/users/5c9ecc4df15d40c386bd4c1e1066b046/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:53.761532 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52677ee-75d8-4527-88c2-c549b59118d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5dab1c4dd4b4973ac2630ba9519f983, user_id=5c9ecc4df15d40c386bd4c1e1066b046, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:53.764387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52677ee-75d8-4527-88c2-c549b59118d6 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:53.862885 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1069/2169] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:53 2022] PUT /identity/v3/projects/e5dab1c4dd4b4973ac2630ba9519f983/users/5c9ecc4df15d40c386bd4c1e1066b046/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:53.867072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a1573-189b-424c-bdc1-09464d3a695a admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:53.879436 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a1573-189b-424c-bdc1-09464d3a695a admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:53.933839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52677ee-75d8-4527-88c2-c549b59118d6 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:53.936134 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c52677ee-75d8-4527-88c2-c549b59118d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:53.943917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190d5d9-0551-4988-9ff4-cb0840660e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:53.944456 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190d5d9-0551-4988-9ff4-cb0840660e71 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:53.944738 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190d5d9-0551-4988-9ff4-cb0840660e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:53.945008 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190d5d9-0551-4988-9ff4-cb0840660e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:53.945658 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190d5d9-0551-4988-9ff4-cb0840660e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:53.947359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190d5d9-0551-4988-9ff4-cb0840660e71 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:53.958339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1101/2170] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:53.962355 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190d5d9-0551-4988-9ff4-cb0840660e71 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:53.974576 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190d5d9-0551-4988-9ff4-cb0840660e71 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:53.983734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54615a17-e836-4927-be16-45a5cf77776e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:53.985380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54615a17-e836-4927-be16-45a5cf77776e admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:53.985665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54615a17-e836-4927-be16-45a5cf77776e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:53.985942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54615a17-e836-4927-be16-45a5cf77776e admin admin] PATH_INFO: `/v3/projects/e5dab1c4dd4b4973ac2630ba9519f983/users/5c9ecc4df15d40c386bd4c1e1066b046/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:53.988640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54615a17-e836-4927-be16-45a5cf77776e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5dab1c4dd4b4973ac2630ba9519f983, user_id=5c9ecc4df15d40c386bd4c1e1066b046, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:53.991630 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54615a17-e836-4927-be16-45a5cf77776e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:54.002498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1102/2171] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:53 2022] PUT /identity/v3/projects/e5dab1c4dd4b4973ac2630ba9519f983/users/5c9ecc4df15d40c386bd4c1e1066b046/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:54.012380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef17ce8-17c7-479c-ba62-f21cbfb7d8dc None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:54.012668 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef17ce8-17c7-479c-ba62-f21cbfb7d8dc None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:54.012945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef17ce8-17c7-479c-ba62-f21cbfb7d8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:54.023894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52677ee-75d8-4527-88c2-c549b59118d6 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:54.042836 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-fef17ce8-17c7-479c-ba62-f21cbfb7d8dc None None] MFA Rules not processed for user `5c9ecc4df15d40c386bd4c1e1066b046`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:54.044098 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-fef17ce8-17c7-479c-ba62-f21cbfb7d8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:54.085038 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:54.085825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:54.086205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:54.086579 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a admin admin] PATH_INFO: `/v3/projects/07cfda046f564ce881b1f0590beb90ef` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:54.087802 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a admin admin] RBAC: Authorizing `identity:delete_project(project_id=07cfda046f564ce881b1f0590beb90ef)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:54.089951 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:54.118575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1103/2172] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:54.130481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef17ce8-17c7-479c-ba62-f21cbfb7d8dc None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:54.132110 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-fef17ce8-17c7-479c-ba62-f21cbfb7d8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:54.191801 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef17ce8-17c7-479c-ba62-f21cbfb7d8dc None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:54.195816 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059e89c-944e-41dd-88dd-4db78e5efd13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:54.196485 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059e89c-944e-41dd-88dd-4db78e5efd13 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:54.196784 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059e89c-944e-41dd-88dd-4db78e5efd13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:54.197043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059e89c-944e-41dd-88dd-4db78e5efd13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:54.206509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059e89c-944e-41dd-88dd-4db78e5efd13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:54.207974 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059e89c-944e-41dd-88dd-4db78e5efd13 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:54.217653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1104/2173] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:36:54.281923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07cfda046f564ce881b1f0590beb90ef', 'request_id': 'req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:54.288856 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07cfda046f564ce881b1f0590beb90ef', 'request_id': 'req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:54.335567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059e89c-944e-41dd-88dd-4db78e5efd13 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:54.344820 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059e89c-944e-41dd-88dd-4db78e5efd13 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:54.404804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07cfda046f564ce881b1f0590beb90ef', 'request_id': 'req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:54.418557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188ca3e-f5f6-4e9b-af80-b1cd23a604cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:54.418557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188ca3e-f5f6-4e9b-af80-b1cd23a604cc service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:54.418557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188ca3e-f5f6-4e9b-af80-b1cd23a604cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:54.418557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188ca3e-f5f6-4e9b-af80-b1cd23a604cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:54.418928 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-1188ca3e-f5f6-4e9b-af80-b1cd23a604cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:54.419015 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a admin admin] The token cache is being invalidate because project 07cfda046f564ce881b1f0590beb90ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:54.419015 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:54.419015 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:54.420610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1070/2174] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:53 2022] DELETE /identity/v3/projects/07cfda046f564ce881b1f0590beb90ef => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:54.424731 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:54.427669 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:54.494010 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188ca3e-f5f6-4e9b-af80-b1cd23a604cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:54.495531 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188ca3e-f5f6-4e9b-af80-b1cd23a604cc service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:54.501267 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1d14f-5080-4ed3-9b8c-bf66500f635a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:54.505881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1105/2175] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:36:54.506589 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425dc4a7-54fc-41bb-8aa8-47d249f7f1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:54.507256 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425dc4a7-54fc-41bb-8aa8-47d249f7f1f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:54.508171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425dc4a7-54fc-41bb-8aa8-47d249f7f1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:54.508496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425dc4a7-54fc-41bb-8aa8-47d249f7f1f4 admin admin] PATH_INFO: `/v3/users/89b52086d30c4305a3d506d128df86a8` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:54.509802 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425dc4a7-54fc-41bb-8aa8-47d249f7f1f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89b52086d30c4305a3d506d128df86a8)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:54.511840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425dc4a7-54fc-41bb-8aa8-47d249f7f1f4 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:54.675669 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:54.752823 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-425dc4a7-54fc-41bb-8aa8-47d249f7f1f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89b52086d30c4305a3d506d128df86a8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:54.852202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-425dc4a7-54fc-41bb-8aa8-47d249f7f1f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89b52086d30c4305a3d506d128df86a8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:54.853898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-425dc4a7-54fc-41bb-8aa8-47d249f7f1f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89b52086d30c4305a3d506d128df86a8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:54.876001 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-425dc4a7-54fc-41bb-8aa8-47d249f7f1f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89b52086d30c4305a3d506d128df86a8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:54.883982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-425dc4a7-54fc-41bb-8aa8-47d249f7f1f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89b52086d30c4305a3d506d128df86a8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:54.891203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1071/2176] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:54 2022] DELETE /identity/v3/users/89b52086d30c4305a3d506d128df86a8 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:55.389826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:55.843991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188ca3e-f5f6-4e9b-af80-b1cd23a604cc service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:55.864391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188ca3e-f5f6-4e9b-af80-b1cd23a604cc service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:55.943738 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec07917-421e-499a-ac4d-c05210dcd8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:55.944380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec07917-421e-499a-ac4d-c05210dcd8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:55.944661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec07917-421e-499a-ac4d-c05210dcd8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:55.944939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec07917-421e-499a-ac4d-c05210dcd8f3 admin admin] PATH_INFO: `/v3/projects/809e7784788a4d249d2f467b7bdadf3a` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:55.946071 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec07917-421e-499a-ac4d-c05210dcd8f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=809e7784788a4d249d2f467b7bdadf3a)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:55.948036 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec07917-421e-499a-ac4d-c05210dcd8f3 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:56.097892 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-dec07917-421e-499a-ac4d-c05210dcd8f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '809e7784788a4d249d2f467b7bdadf3a', 'request_id': 'req-dec07917-421e-499a-ac4d-c05210dcd8f3'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:56.105735 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-dec07917-421e-499a-ac4d-c05210dcd8f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '809e7784788a4d249d2f467b7bdadf3a', 'request_id': 'req-dec07917-421e-499a-ac4d-c05210dcd8f3'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:56.201265 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-dec07917-421e-499a-ac4d-c05210dcd8f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '809e7784788a4d249d2f467b7bdadf3a', 'request_id': 'req-dec07917-421e-499a-ac4d-c05210dcd8f3'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:56.217927 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425dc4a7-54fc-41bb-8aa8-47d249f7f1f4 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:56.220165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-425dc4a7-54fc-41bb-8aa8-47d249f7f1f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:56.221721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-dec07917-421e-499a-ac4d-c05210dcd8f3 admin admin] The token cache is being invalidate because project 809e7784788a4d249d2f467b7bdadf3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:56.222081 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-dec07917-421e-499a-ac4d-c05210dcd8f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:56.222968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-dec07917-421e-499a-ac4d-c05210dcd8f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:56.232102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1106/2177] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:55 2022] DELETE /identity/v3/projects/809e7784788a4d249d2f467b7bdadf3a => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:56.233522 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869c420-f054-4b27-b268-b9ded4126d28 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:56.233805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869c420-f054-4b27-b268-b9ded4126d28 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:56.234166 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869c420-f054-4b27-b268-b9ded4126d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:56.236366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425dc4a7-54fc-41bb-8aa8-47d249f7f1f4 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:56.255733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-7869c420-f054-4b27-b268-b9ded4126d28 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:56.257324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7869c420-f054-4b27-b268-b9ded4126d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:56.300424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0ca6d-a0b4-4b55-8488-87c16b1d8116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:56.301067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0ca6d-a0b4-4b55-8488-87c16b1d8116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:56.301537 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0ca6d-a0b4-4b55-8488-87c16b1d8116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:56.302165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0ca6d-a0b4-4b55-8488-87c16b1d8116 admin admin] PATH_INFO: `/v3/users/7a489a31d97d46c4b8e1acbfd8200d79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:56.303693 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0ca6d-a0b4-4b55-8488-87c16b1d8116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a489a31d97d46c4b8e1acbfd8200d79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:56.306711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0ca6d-a0b4-4b55-8488-87c16b1d8116 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:56.340027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1107/2178] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:56.349515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982f9a5-59bd-46ce-ae9c-792e7f660a21 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:56.349630 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982f9a5-59bd-46ce-ae9c-792e7f660a21 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:56.349898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982f9a5-59bd-46ce-ae9c-792e7f660a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:56.372281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-d982f9a5-59bd-46ce-ae9c-792e7f660a21 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:56.380041 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d982f9a5-59bd-46ce-ae9c-792e7f660a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:56.383916 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1108/2179] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:56.387361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982f9a5-59bd-46ce-ae9c-792e7f660a21 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:56.399514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982f9a5-59bd-46ce-ae9c-792e7f660a21 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:56.404591 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919eb79-0578-4239-a03e-f1b8a36aa43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:56.405324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919eb79-0578-4239-a03e-f1b8a36aa43c admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:56.405603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919eb79-0578-4239-a03e-f1b8a36aa43c admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:56.405871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919eb79-0578-4239-a03e-f1b8a36aa43c admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:56.407144 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919eb79-0578-4239-a03e-f1b8a36aa43c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:36:56.407450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919eb79-0578-4239-a03e-f1b8a36aa43c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:56.409074 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919eb79-0578-4239-a03e-f1b8a36aa43c admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:56.423693 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1109/2180] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:36:56.434423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919eb79-0578-4239-a03e-f1b8a36aa43c admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:56.455471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919eb79-0578-4239-a03e-f1b8a36aa43c admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:56.463271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf4edd-4e11-467b-89f0-28a0917269f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:56.463916 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf4edd-4e11-467b-89f0-28a0917269f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:56.464347 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf4edd-4e11-467b-89f0-28a0917269f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:56.464626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf4edd-4e11-467b-89f0-28a0917269f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:56.465436 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf4edd-4e11-467b-89f0-28a0917269f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:56.468966 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf4edd-4e11-467b-89f0-28a0917269f0 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:56.566581 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-d2b0ca6d-a0b4-4b55-8488-87c16b1d8116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a489a31d97d46c4b8e1acbfd8200d79'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:56.588781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1110/2181] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:56 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:36:56.590047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf4edd-4e11-467b-89f0-28a0917269f0 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:56.640130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-d2b0ca6d-a0b4-4b55-8488-87c16b1d8116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a489a31d97d46c4b8e1acbfd8200d79'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:56.640863 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-d2b0ca6d-a0b4-4b55-8488-87c16b1d8116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a489a31d97d46c4b8e1acbfd8200d79'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:56.653809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-d2b0ca6d-a0b4-4b55-8488-87c16b1d8116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a489a31d97d46c4b8e1acbfd8200d79'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:56.653991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf4edd-4e11-467b-89f0-28a0917269f0 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:56.659377 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad8880-7acf-44ea-b857-d7e3681b17c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:56.660039 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad8880-7acf-44ea-b857-d7e3681b17c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:56.660320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad8880-7acf-44ea-b857-d7e3681b17c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:56.660590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad8880-7acf-44ea-b857-d7e3681b17c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:56.661260 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad8880-7acf-44ea-b857-d7e3681b17c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:56.661416 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-d2b0ca6d-a0b4-4b55-8488-87c16b1d8116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a489a31d97d46c4b8e1acbfd8200d79'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:56.663227 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad8880-7acf-44ea-b857-d7e3681b17c5 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:56.677518 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1072/2182] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:56 2022] DELETE /identity/v3/users/7a489a31d97d46c4b8e1acbfd8200d79 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:56.679614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d38ff-c76b-4dae-bb44-e89cbb8393e9 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:56.679614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d38ff-c76b-4dae-bb44-e89cbb8393e9 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:56.679614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d38ff-c76b-4dae-bb44-e89cbb8393e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:56.700485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-693d38ff-c76b-4dae-bb44-e89cbb8393e9 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:56.701698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-693d38ff-c76b-4dae-bb44-e89cbb8393e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:56.770879 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1073/2183] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:56.779378 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d38ff-c76b-4dae-bb44-e89cbb8393e9 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:56.780775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-693d38ff-c76b-4dae-bb44-e89cbb8393e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:56.801705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1111/2184] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:56 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:36:56.806037 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad8880-7acf-44ea-b857-d7e3681b17c5 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:56.809751 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-faad8880-7acf-44ea-b857-d7e3681b17c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:56.837313 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad8880-7acf-44ea-b857-d7e3681b17c5 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:56.848967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb0d32-b442-4af0-a639-43126289481c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:56.849652 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb0d32-b442-4af0-a639-43126289481c admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:56.849948 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb0d32-b442-4af0-a639-43126289481c admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:56.850378 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb0d32-b442-4af0-a639-43126289481c admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:56.851073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb0d32-b442-4af0-a639-43126289481c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:56.853073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb0d32-b442-4af0-a639-43126289481c admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:56.867005 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1112/2185] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:56.867243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d38ff-c76b-4dae-bb44-e89cbb8393e9 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:56.868221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb0d32-b442-4af0-a639-43126289481c admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:56.871711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae358458-8bdd-4f04-b6a6-26b13f521298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:56.872424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae358458-8bdd-4f04-b6a6-26b13f521298 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:56.872778 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae358458-8bdd-4f04-b6a6-26b13f521298 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:56.873123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae358458-8bdd-4f04-b6a6-26b13f521298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:56.881054 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb0d32-b442-4af0-a639-43126289481c admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:56.883855 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae358458-8bdd-4f04-b6a6-26b13f521298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:56.885376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae358458-8bdd-4f04-b6a6-26b13f521298 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:56.886170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39ea4a-7afc-4c38-94b7-f78c789c7caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:56.886801 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39ea4a-7afc-4c38-94b7-f78c789c7caa admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:56.887114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39ea4a-7afc-4c38-94b7-f78c789c7caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:56.887394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39ea4a-7afc-4c38-94b7-f78c789c7caa admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:56.888042 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39ea4a-7afc-4c38-94b7-f78c789c7caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:56.889776 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39ea4a-7afc-4c38-94b7-f78c789c7caa admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:56.896331 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1074/2186] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:36:56.905506 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1113/2187] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:56.908131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae358458-8bdd-4f04-b6a6-26b13f521298 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:56.920897 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae358458-8bdd-4f04-b6a6-26b13f521298 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:56.930162 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875582b-6ebd-47a4-9410-bddcee50929b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:56.930874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875582b-6ebd-47a4-9410-bddcee50929b admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:56.931712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875582b-6ebd-47a4-9410-bddcee50929b admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:56.932000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875582b-6ebd-47a4-9410-bddcee50929b admin admin] PATH_INFO: `/v3/projects/bbc4e17035aa4a1a9188f1177b92e089/users/571c31c7960a401f89c98a179db2f08e/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:56.948996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c875582b-6ebd-47a4-9410-bddcee50929b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbc4e17035aa4a1a9188f1177b92e089, user_id=571c31c7960a401f89c98a179db2f08e, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:56.951821 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c875582b-6ebd-47a4-9410-bddcee50929b admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:57.082099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1075/2188] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:56 2022] PUT /identity/v3/projects/bbc4e17035aa4a1a9188f1177b92e089/users/571c31c7960a401f89c98a179db2f08e/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:57.093809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71f990-19e0-4969-a46f-9f4a69d996ec None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:57.094116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71f990-19e0-4969-a46f-9f4a69d996ec None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:57.094372 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71f990-19e0-4969-a46f-9f4a69d996ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:57.128181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-ac71f990-19e0-4969-a46f-9f4a69d996ec None None] MFA Rules not processed for user `571c31c7960a401f89c98a179db2f08e`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:57.133627 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ac71f990-19e0-4969-a46f-9f4a69d996ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:57.207416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1114/2189] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:57.212127 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875582b-6ebd-47a4-9410-bddcee50929b admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:57.292962 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875582b-6ebd-47a4-9410-bddcee50929b admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:57.313993 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d85c7-08eb-49c8-b48b-1a191fb086cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:57.315123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d85c7-08eb-49c8-b48b-1a191fb086cd admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:57.315402 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d85c7-08eb-49c8-b48b-1a191fb086cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:57.315706 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d85c7-08eb-49c8-b48b-1a191fb086cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:57.316358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d85c7-08eb-49c8-b48b-1a191fb086cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:57.318390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d85c7-08eb-49c8-b48b-1a191fb086cd admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:57.372358 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:57.409991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71f990-19e0-4969-a46f-9f4a69d996ec None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:57.412005 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ac71f990-19e0-4969-a46f-9f4a69d996ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:57.429403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71f990-19e0-4969-a46f-9f4a69d996ec None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:57.434426 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f347d-37b6-43df-95a2-5591b1d89e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:57.435246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f347d-37b6-43df-95a2-5591b1d89e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:57.435528 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f347d-37b6-43df-95a2-5591b1d89e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:57.435801 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f347d-37b6-43df-95a2-5591b1d89e0c admin admin] PATH_INFO: `/v3/projects/3bd167e7a5a84dedb7d534ee8eefcb0f` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:57.437505 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f347d-37b6-43df-95a2-5591b1d89e0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd167e7a5a84dedb7d534ee8eefcb0f)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:57.440963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f347d-37b6-43df-95a2-5591b1d89e0c admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:57.450765 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1076/2190] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:57 2022] POST /identity/v3/projects => generated 385 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:57.451813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d85c7-08eb-49c8-b48b-1a191fb086cd admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:57.534847 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d85c7-08eb-49c8-b48b-1a191fb086cd admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:57.542763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb382c30-e44c-4efb-94c4-21a96b287ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:57.543645 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb382c30-e44c-4efb-94c4-21a96b287ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:57.544113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb382c30-e44c-4efb-94c4-21a96b287ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:57.544586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb382c30-e44c-4efb-94c4-21a96b287ee7 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:57.545400 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb382c30-e44c-4efb-94c4-21a96b287ee7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:57.548339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb382c30-e44c-4efb-94c4-21a96b287ee7 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:57.559032 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-994f347d-37b6-43df-95a2-5591b1d89e0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd167e7a5a84dedb7d534ee8eefcb0f', 'request_id': 'req-994f347d-37b6-43df-95a2-5591b1d89e0c'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:57.565800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-994f347d-37b6-43df-95a2-5591b1d89e0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd167e7a5a84dedb7d534ee8eefcb0f', 'request_id': 'req-994f347d-37b6-43df-95a2-5591b1d89e0c'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:57.686986 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1077/2191] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:57 2022] POST /identity/v3/users => generated 299 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:57.690824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb382c30-e44c-4efb-94c4-21a96b287ee7 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:57.701122 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb382c30-e44c-4efb-94c4-21a96b287ee7 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:57.708935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-994f347d-37b6-43df-95a2-5591b1d89e0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd167e7a5a84dedb7d534ee8eefcb0f', 'request_id': 'req-994f347d-37b6-43df-95a2-5591b1d89e0c'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:57.720846 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-994f347d-37b6-43df-95a2-5591b1d89e0c admin admin] The token cache is being invalidate because project 3bd167e7a5a84dedb7d534ee8eefcb0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:36:57.721132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-994f347d-37b6-43df-95a2-5591b1d89e0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:57.721764 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-994f347d-37b6-43df-95a2-5591b1d89e0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:36:57.723515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1115/2192] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:36:57 2022] DELETE /identity/v3/projects/3bd167e7a5a84dedb7d534ee8eefcb0f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:57.733122 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f032f-54ea-4504-8769-d859ce977a51 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:57.733411 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f032f-54ea-4504-8769-d859ce977a51 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:57.733771 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f032f-54ea-4504-8769-d859ce977a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:57.760422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-cb1f032f-54ea-4504-8769-d859ce977a51 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:57.761730 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-cb1f032f-54ea-4504-8769-d859ce977a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:57.763054 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4b09b-71ff-4df5-b68b-44dbe690a6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:57.763693 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4b09b-71ff-4df5-b68b-44dbe690a6c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:57.763960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4b09b-71ff-4df5-b68b-44dbe690a6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:57.778096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4b09b-71ff-4df5-b68b-44dbe690a6c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:57.778096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4b09b-71ff-4df5-b68b-44dbe690a6c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:57.778096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4b09b-71ff-4df5-b68b-44dbe690a6c4 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:57.778395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1116/2193] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:57.778504 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1078/2194] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:57.779772 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7483a7-6c84-47dc-bb38-3a2500c6efd7 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:57.780125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7483a7-6c84-47dc-bb38-3a2500c6efd7 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:57.780403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7483a7-6c84-47dc-bb38-3a2500c6efd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:57.783751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4b09b-71ff-4df5-b68b-44dbe690a6c4 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:57.785619 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d1b4b09b-71ff-4df5-b68b-44dbe690a6c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:57.798750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4b09b-71ff-4df5-b68b-44dbe690a6c4 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:57.801713 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:57.802489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-7e7483a7-6c84-47dc-bb38-3a2500c6efd7 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:57.803711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7e7483a7-6c84-47dc-bb38-3a2500c6efd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:57.805399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1b1cd-b704-4ba8-be89-2ca288eb6329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:57.806071 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1b1cd-b704-4ba8-be89-2ca288eb6329 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:57.806451 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1b1cd-b704-4ba8-be89-2ca288eb6329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:57.806761 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1b1cd-b704-4ba8-be89-2ca288eb6329 admin admin] PATH_INFO: `/v3/projects/8141b2cdbd0e4cd186b8e22b2786a673/users/9b2f32643f0d410caca6a24c1446b982/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:57.820414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1117/2195] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:57.824685 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7483a7-6c84-47dc-bb38-3a2500c6efd7 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:57.824857 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1b1cd-b704-4ba8-be89-2ca288eb6329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8141b2cdbd0e4cd186b8e22b2786a673, user_id=9b2f32643f0d410caca6a24c1446b982, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:57.828456 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1b1cd-b704-4ba8-be89-2ca288eb6329 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:57.835841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7483a7-6c84-47dc-bb38-3a2500c6efd7 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:57.841233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114fa04-9e4d-4f95-ba03-82c6bcb48803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:57.844299 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114fa04-9e4d-4f95-ba03-82c6bcb48803 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:57.844299 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114fa04-9e4d-4f95-ba03-82c6bcb48803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:57.844299 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114fa04-9e4d-4f95-ba03-82c6bcb48803 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:57.844299 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114fa04-9e4d-4f95-ba03-82c6bcb48803 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:36:57.844299 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114fa04-9e4d-4f95-ba03-82c6bcb48803 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:57.845569 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114fa04-9e4d-4f95-ba03-82c6bcb48803 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:57.857535 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1118/2196] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:36:57.862601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114fa04-9e4d-4f95-ba03-82c6bcb48803 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:57.874078 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114fa04-9e4d-4f95-ba03-82c6bcb48803 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:57.888343 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859e66c-037d-4e62-a31c-0bb01a94f9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:57.889052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859e66c-037d-4e62-a31c-0bb01a94f9c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:57.889336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859e66c-037d-4e62-a31c-0bb01a94f9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:57.889613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859e66c-037d-4e62-a31c-0bb01a94f9c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:57.890249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859e66c-037d-4e62-a31c-0bb01a94f9c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:57.892658 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859e66c-037d-4e62-a31c-0bb01a94f9c1 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:57.913976 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1079/2197] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:57 2022] PUT /identity/v3/projects/8141b2cdbd0e4cd186b8e22b2786a673/users/9b2f32643f0d410caca6a24c1446b982/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:57.920355 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1b1cd-b704-4ba8-be89-2ca288eb6329 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:57.988570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1b1cd-b704-4ba8-be89-2ca288eb6329 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:57.993756 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca06c8-d5e1-49fc-9f60-e7a213a85b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:57.994400 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca06c8-d5e1-49fc-9f60-e7a213a85b7a admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:57.994682 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca06c8-d5e1-49fc-9f60-e7a213a85b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:57.995008 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca06c8-d5e1-49fc-9f60-e7a213a85b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:57.995823 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca06c8-d5e1-49fc-9f60-e7a213a85b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:57.997894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca06c8-d5e1-49fc-9f60-e7a213a85b7a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:58.011394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1080/2198] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:58.012935 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca06c8-d5e1-49fc-9f60-e7a213a85b7a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:58.024145 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1119/2199] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:57 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:36:58.025309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca06c8-d5e1-49fc-9f60-e7a213a85b7a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:58.028581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859e66c-037d-4e62-a31c-0bb01a94f9c1 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:58.122829 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3c7ab-70d7-431a-b87f-97732f092697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:58.122829 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3c7ab-70d7-431a-b87f-97732f092697 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:58.123178 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3c7ab-70d7-431a-b87f-97732f092697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:58.123178 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3c7ab-70d7-431a-b87f-97732f092697 admin admin] PATH_INFO: `/v3/projects/8141b2cdbd0e4cd186b8e22b2786a673/users/9b2f32643f0d410caca6a24c1446b982/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:58.126665 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3c7ab-70d7-431a-b87f-97732f092697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8141b2cdbd0e4cd186b8e22b2786a673, user_id=9b2f32643f0d410caca6a24c1446b982, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:58.129522 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3c7ab-70d7-431a-b87f-97732f092697 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:58.138732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859e66c-037d-4e62-a31c-0bb01a94f9c1 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:58.142835 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1081/2200] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:58 2022] PUT /identity/v3/projects/8141b2cdbd0e4cd186b8e22b2786a673/users/9b2f32643f0d410caca6a24c1446b982/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:36:58.153392 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f91e5-e197-4a68-b779-b0cf4d822c18 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:58.153695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f91e5-e197-4a68-b779-b0cf4d822c18 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:58.153969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f91e5-e197-4a68-b779-b0cf4d822c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:58.204890 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-3e3f91e5-e197-4a68-b779-b0cf4d822c18 None None] MFA Rules not processed for user `9b2f32643f0d410caca6a24c1446b982`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:58.206254 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-3e3f91e5-e197-4a68-b779-b0cf4d822c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:58.207785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ca5a7-f887-4618-ac42-594aabf5ce21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:58.208634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca5a7-f887-4618-ac42-594aabf5ce21 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:58.209120 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca5a7-f887-4618-ac42-594aabf5ce21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:58.209736 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca5a7-f887-4618-ac42-594aabf5ce21 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:58.211394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ca5a7-f887-4618-ac42-594aabf5ce21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:58.214467 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ca5a7-f887-4618-ac42-594aabf5ce21 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:58.283308 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1082/2201] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:36:58.298363 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f91e5-e197-4a68-b779-b0cf4d822c18 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:58.300220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-3e3f91e5-e197-4a68-b779-b0cf4d822c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:58.351076 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1120/2202] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:58 2022] POST /identity/v3/users => generated 289 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:58.362411 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ca5a7-f887-4618-ac42-594aabf5ce21 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:58.373130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ca5a7-f887-4618-ac42-594aabf5ce21 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:58.378762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082acc2-bf3f-447b-9a2e-386897595db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:58.379458 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082acc2-bf3f-447b-9a2e-386897595db3 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:58.379738 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082acc2-bf3f-447b-9a2e-386897595db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:58.380036 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082acc2-bf3f-447b-9a2e-386897595db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:58.385635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082acc2-bf3f-447b-9a2e-386897595db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:58.385635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082acc2-bf3f-447b-9a2e-386897595db3 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:58.392588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1121/2203] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:58.396491 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082acc2-bf3f-447b-9a2e-386897595db3 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:58.400621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f91e5-e197-4a68-b779-b0cf4d822c18 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:58.405285 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a147bb-95fd-484c-883e-332403a5ca04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:58.406545 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a147bb-95fd-484c-883e-332403a5ca04 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:58.406964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a147bb-95fd-484c-883e-332403a5ca04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:58.407333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a147bb-95fd-484c-883e-332403a5ca04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:58.410512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082acc2-bf3f-447b-9a2e-386897595db3 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:58.417167 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51156ede-2819-41cd-a6c0-994be5f6bd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:58.417336 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a147bb-95fd-484c-883e-332403a5ca04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:58.417919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51156ede-2819-41cd-a6c0-994be5f6bd46 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:58.418220 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51156ede-2819-41cd-a6c0-994be5f6bd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:58.418502 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51156ede-2819-41cd-a6c0-994be5f6bd46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:58.419238 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a147bb-95fd-484c-883e-332403a5ca04 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:58.419351 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51156ede-2819-41cd-a6c0-994be5f6bd46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:58.420874 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51156ede-2819-41cd-a6c0-994be5f6bd46 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:58.438171 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1122/2204] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:58.446143 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51156ede-2819-41cd-a6c0-994be5f6bd46 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:58.446340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1083/2205] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:36:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:36:58.451700 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63887c2e-0747-4e6c-be0e-5c510cd1ae04 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:58.452131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63887c2e-0747-4e6c-be0e-5c510cd1ae04 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:58.452508 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63887c2e-0747-4e6c-be0e-5c510cd1ae04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:58.459268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51156ede-2819-41cd-a6c0-994be5f6bd46 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:58.472801 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae3ca0-91d5-4d9d-9c7b-948d2be2e6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:58.473084 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae3ca0-91d5-4d9d-9c7b-948d2be2e6b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:58.473382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae3ca0-91d5-4d9d-9c7b-948d2be2e6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:58.473904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae3ca0-91d5-4d9d-9c7b-948d2be2e6b3 admin admin] PATH_INFO: `/v3/projects/1ae8c0838b56416e97b99fb71c53fe5b/users/675a42d11a1a4931a36bf223633d5a04/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:58.479513 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-63887c2e-0747-4e6c-be0e-5c510cd1ae04 None None] MFA Rules not processed for user `e8429576e286422cb09591897e82cd69`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:58.480845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-63887c2e-0747-4e6c-be0e-5c510cd1ae04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:58.489626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae3ca0-91d5-4d9d-9c7b-948d2be2e6b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae8c0838b56416e97b99fb71c53fe5b, user_id=675a42d11a1a4931a36bf223633d5a04, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:58.492410 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae3ca0-91d5-4d9d-9c7b-948d2be2e6b3 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:58.535581 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1084/2206] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:58.595508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1123/2207] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:58 2022] PUT /identity/v3/projects/1ae8c0838b56416e97b99fb71c53fe5b/users/675a42d11a1a4931a36bf223633d5a04/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:58.611111 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65111cc8-5930-41ec-97f6-20606a4655a9 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:58.611403 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65111cc8-5930-41ec-97f6-20606a4655a9 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:58.623026 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65111cc8-5930-41ec-97f6-20606a4655a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:58.657166 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-65111cc8-5930-41ec-97f6-20606a4655a9 None None] MFA Rules not processed for user `675a42d11a1a4931a36bf223633d5a04`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:58.658486 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-65111cc8-5930-41ec-97f6-20606a4655a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:58.724144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1085/2208] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:58.728443 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae3ca0-91d5-4d9d-9c7b-948d2be2e6b3 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:58.801554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae3ca0-91d5-4d9d-9c7b-948d2be2e6b3 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:58.806755 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372719c8-78ab-4840-81b8-ab4e514b7c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:58.807401 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372719c8-78ab-4840-81b8-ab4e514b7c8b admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:58.807687 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372719c8-78ab-4840-81b8-ab4e514b7c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:58.808042 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372719c8-78ab-4840-81b8-ab4e514b7c8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:58.808907 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372719c8-78ab-4840-81b8-ab4e514b7c8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:58.811147 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372719c8-78ab-4840-81b8-ab4e514b7c8b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:58.915696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1124/2209] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:58 2022] POST /identity/v3/projects => generated 377 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:58.919836 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65111cc8-5930-41ec-97f6-20606a4655a9 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:58.934903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65111cc8-5930-41ec-97f6-20606a4655a9 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:59.001574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1c599-1fa9-491b-b62e-e5e7a1794eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:59.002360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1c599-1fa9-491b-b62e-e5e7a1794eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:59.002648 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1c599-1fa9-491b-b62e-e5e7a1794eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:59.002986 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1c599-1fa9-491b-b62e-e5e7a1794eb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:59.003838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1c599-1fa9-491b-b62e-e5e7a1794eb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:59.005821 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1c599-1fa9-491b-b62e-e5e7a1794eb8 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:59.163114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1086/2210] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:58 2022] POST /identity/v3/users => generated 289 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:59.167111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372719c8-78ab-4840-81b8-ab4e514b7c8b admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:59.177713 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372719c8-78ab-4840-81b8-ab4e514b7c8b admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:59.182581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433fe31-491c-4cb9-b12e-27a24a4fa093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:59.183338 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433fe31-491c-4cb9-b12e-27a24a4fa093 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:59.183621 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433fe31-491c-4cb9-b12e-27a24a4fa093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:59.183889 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433fe31-491c-4cb9-b12e-27a24a4fa093 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:59.184525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433fe31-491c-4cb9-b12e-27a24a4fa093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:59.186186 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433fe31-491c-4cb9-b12e-27a24a4fa093 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:59.208216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1125/2211] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:59.212266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1c599-1fa9-491b-b62e-e5e7a1794eb8 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:59.223176 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1c599-1fa9-491b-b62e-e5e7a1794eb8 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:59.228227 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302bce1-99bd-4bac-88a4-f9b0cd6dc3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:59.229179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302bce1-99bd-4bac-88a4-f9b0cd6dc3ea admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:59.229617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302bce1-99bd-4bac-88a4-f9b0cd6dc3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:59.230053 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302bce1-99bd-4bac-88a4-f9b0cd6dc3ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:59.230905 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302bce1-99bd-4bac-88a4-f9b0cd6dc3ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:59.232686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302bce1-99bd-4bac-88a4-f9b0cd6dc3ea admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:59.250750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1087/2212] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:36:59.251327 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433fe31-491c-4cb9-b12e-27a24a4fa093 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:59.261806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433fe31-491c-4cb9-b12e-27a24a4fa093 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:59.267490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc68f4-a83f-4496-b3ce-ec0e2b72c8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:59.268162 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc68f4-a83f-4496-b3ce-ec0e2b72c8ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:59.268445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc68f4-a83f-4496-b3ce-ec0e2b72c8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:59.268754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc68f4-a83f-4496-b3ce-ec0e2b72c8ad admin admin] PATH_INFO: `/v3/projects/5aaa593cc8f046dabf2284143438326b/users/1e6d86d2b0854fb99168b7b1879a1b79/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:59.284872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc68f4-a83f-4496-b3ce-ec0e2b72c8ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aaa593cc8f046dabf2284143438326b, user_id=1e6d86d2b0854fb99168b7b1879a1b79, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:59.287539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc68f4-a83f-4496-b3ce-ec0e2b72c8ad admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:59.363931 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1126/2213] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:59 2022] PUT /identity/v3/projects/5aaa593cc8f046dabf2284143438326b/users/1e6d86d2b0854fb99168b7b1879a1b79/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:36:59.373991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782272b-6c81-44c5-a9a0-564a5f698739 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:59.374297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782272b-6c81-44c5-a9a0-564a5f698739 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:59.374592 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782272b-6c81-44c5-a9a0-564a5f698739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:59.409556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-5782272b-6c81-44c5-a9a0-564a5f698739 None None] MFA Rules not processed for user `1e6d86d2b0854fb99168b7b1879a1b79`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:36:59.411028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-5782272b-6c81-44c5-a9a0-564a5f698739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:36:59.440991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:59.497463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1088/2214] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:36:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:36:59.503002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc68f4-a83f-4496-b3ce-ec0e2b72c8ad admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:59.568741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc68f4-a83f-4496-b3ce-ec0e2b72c8ad admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:59.573791 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610d4ee-f2f8-4888-bd9f-d31f8e6a27ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:59.574429 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610d4ee-f2f8-4888-bd9f-d31f8e6a27ad admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:59.574742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610d4ee-f2f8-4888-bd9f-d31f8e6a27ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:59.575072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610d4ee-f2f8-4888-bd9f-d31f8e6a27ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:59.575716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610d4ee-f2f8-4888-bd9f-d31f8e6a27ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:59.577602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610d4ee-f2f8-4888-bd9f-d31f8e6a27ad admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:59.704672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1127/2215] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:36:59 2022] POST /identity/v3/projects => generated 375 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:36:59.708826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782272b-6c81-44c5-a9a0-564a5f698739 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:59.719688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782272b-6c81-44c5-a9a0-564a5f698739 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:59.775776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d463e-0e09-4c7e-a5ba-519ae9601fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:59.776377 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d463e-0e09-4c7e-a5ba-519ae9601fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:59.776377 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d463e-0e09-4c7e-a5ba-519ae9601fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:59.776566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d463e-0e09-4c7e-a5ba-519ae9601fe6 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:59.777195 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d463e-0e09-4c7e-a5ba-519ae9601fe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:59.779179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d463e-0e09-4c7e-a5ba-519ae9601fe6 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:59.792754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610d4ee-f2f8-4888-bd9f-d31f8e6a27ad admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:59.800483 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:36:59.803771 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610d4ee-f2f8-4888-bd9f-d31f8e6a27ad admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:59.808527 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbbedb-a91e-4f01-b476-8cba6aca2ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:59.809150 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbbedb-a91e-4f01-b476-8cba6aca2ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:59.809426 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbbedb-a91e-4f01-b476-8cba6aca2ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:59.809697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbbedb-a91e-4f01-b476-8cba6aca2ae9 admin admin] PATH_INFO: `/v3/users/571c31c7960a401f89c98a179db2f08e` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:59.810847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abbbedb-a91e-4f01-b476-8cba6aca2ae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=571c31c7960a401f89c98a179db2f08e)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:59.813394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abbbedb-a91e-4f01-b476-8cba6aca2ae9 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:59.863943 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1089/2216] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:36:59 2022] POST /identity/v3/users => generated 294 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:36:59.867974 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d463e-0e09-4c7e-a5ba-519ae9601fe6 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:59.877819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d463e-0e09-4c7e-a5ba-519ae9601fe6 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:59.882481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffc024-9e08-4a59-abc0-06dc19442dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:59.883098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffc024-9e08-4a59-abc0-06dc19442dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:59.883383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffc024-9e08-4a59-abc0-06dc19442dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:59.883659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffc024-9e08-4a59-abc0-06dc19442dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:59.884294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ffc024-9e08-4a59-abc0-06dc19442dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:59.885874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ffc024-9e08-4a59-abc0-06dc19442dc2 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:36:59.896136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1090/2217] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:36:59.900072 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffc024-9e08-4a59-abc0-06dc19442dc2 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:36:59.909977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffc024-9e08-4a59-abc0-06dc19442dc2 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:36:59.914752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330be0b-0b2b-441c-b0e8-6997545f81ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:36:59.915385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330be0b-0b2b-441c-b0e8-6997545f81ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:36:59.915679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330be0b-0b2b-441c-b0e8-6997545f81ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:36:59.915959 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330be0b-0b2b-441c-b0e8-6997545f81ac admin admin] PATH_INFO: `/v3/projects/b252456d95d94021804c380e4b537113/users/524576446f8e46eeaf373a32d03db77b/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:36:59.930662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330be0b-0b2b-441c-b0e8-6997545f81ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b252456d95d94021804c380e4b537113, user_id=524576446f8e46eeaf373a32d03db77b, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:36:59.933345 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330be0b-0b2b-441c-b0e8-6997545f81ac admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:00.063973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1091/2218] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:36:59 2022] PUT /identity/v3/projects/b252456d95d94021804c380e4b537113/users/524576446f8e46eeaf373a32d03db77b/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:37:00.067777 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330be0b-0b2b-441c-b0e8-6997545f81ac admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:00.109600 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-1abbbedb-a91e-4f01-b476-8cba6aca2ae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '571c31c7960a401f89c98a179db2f08e'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:00.131818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330be0b-0b2b-441c-b0e8-6997545f81ac admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:00.136751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba78e6c-3400-49f6-b9fc-377527ea73b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:00.137212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba78e6c-3400-49f6-b9fc-377527ea73b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:00.137506 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba78e6c-3400-49f6-b9fc-377527ea73b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:00.137777 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba78e6c-3400-49f6-b9fc-377527ea73b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:00.138415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba78e6c-3400-49f6-b9fc-377527ea73b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:00.140059 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba78e6c-3400-49f6-b9fc-377527ea73b9 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:00.153432 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1092/2219] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:37:00.156182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba78e6c-3400-49f6-b9fc-377527ea73b9 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:00.165921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba78e6c-3400-49f6-b9fc-377527ea73b9 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:00.170669 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa4eb2-d180-40a0-9b3a-e582385bf068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:00.171409 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa4eb2-d180-40a0-9b3a-e582385bf068 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:00.171695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa4eb2-d180-40a0-9b3a-e582385bf068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:00.171970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa4eb2-d180-40a0-9b3a-e582385bf068 admin admin] PATH_INFO: `/v3/projects/b252456d95d94021804c380e4b537113/users/524576446f8e46eeaf373a32d03db77b/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:00.173885 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa4eb2-d180-40a0-9b3a-e582385bf068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b252456d95d94021804c380e4b537113, user_id=524576446f8e46eeaf373a32d03db77b, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:00.176577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa4eb2-d180-40a0-9b3a-e582385bf068 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:00.186719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1093/2220] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:37:00 2022] PUT /identity/v3/projects/b252456d95d94021804c380e4b537113/users/524576446f8e46eeaf373a32d03db77b/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:37:00.212366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-1abbbedb-a91e-4f01-b476-8cba6aca2ae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '571c31c7960a401f89c98a179db2f08e'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:00.227998 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-1abbbedb-a91e-4f01-b476-8cba6aca2ae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '571c31c7960a401f89c98a179db2f08e'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:00.228745 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-1abbbedb-a91e-4f01-b476-8cba6aca2ae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '571c31c7960a401f89c98a179db2f08e'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:00.237029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-1abbbedb-a91e-4f01-b476-8cba6aca2ae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '571c31c7960a401f89c98a179db2f08e'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:00.243566 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1128/2221] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:36:59 2022] DELETE /identity/v3/users/571c31c7960a401f89c98a179db2f08e => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:00.248215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a645df-e831-4cd0-b085-bcb28af5038a None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:00.248502 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a645df-e831-4cd0-b085-bcb28af5038a None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:00.248771 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a645df-e831-4cd0-b085-bcb28af5038a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:00.258040 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda0982-8f57-455e-a5e8-347220b1c54f None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:00.258331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda0982-8f57-455e-a5e8-347220b1c54f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:00.258606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda0982-8f57-455e-a5e8-347220b1c54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:00.269028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-63a645df-e831-4cd0-b085-bcb28af5038a None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:00.270253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-63a645df-e831-4cd0-b085-bcb28af5038a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:00.287240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-eeda0982-8f57-455e-a5e8-347220b1c54f None None] MFA Rules not processed for user `524576446f8e46eeaf373a32d03db77b`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:00.288433 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-eeda0982-8f57-455e-a5e8-347220b1c54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:00.344257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1094/2222] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:37:00.345694 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a645df-e831-4cd0-b085-bcb28af5038a None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:00.348292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-63a645df-e831-4cd0-b085-bcb28af5038a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:00.365313 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a645df-e831-4cd0-b085-bcb28af5038a None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:00.371767 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d2a76-a111-47d2-a82d-4deeca0c4c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:00.372659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d2a76-a111-47d2-a82d-4deeca0c4c0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:00.373043 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d2a76-a111-47d2-a82d-4deeca0c4c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:00.373421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d2a76-a111-47d2-a82d-4deeca0c4c0b admin admin] PATH_INFO: `/v3/users/ed240a780e2942eaafbc7b1aa33b1d0c` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:00.375020 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d2a76-a111-47d2-a82d-4deeca0c4c0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed240a780e2942eaafbc7b1aa33b1d0c)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:00.377275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d2a76-a111-47d2-a82d-4deeca0c4c0b admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:00.381963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1129/2223] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:37:00.383072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda0982-8f57-455e-a5e8-347220b1c54f None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:00.384390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-eeda0982-8f57-455e-a5e8-347220b1c54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:00.447287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda0982-8f57-455e-a5e8-347220b1c54f None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:00.450977 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46729547-0981-4dab-8184-e0630fc3a757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:00.451645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46729547-0981-4dab-8184-e0630fc3a757 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:00.451933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46729547-0981-4dab-8184-e0630fc3a757 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:00.452206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46729547-0981-4dab-8184-e0630fc3a757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:00.461001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46729547-0981-4dab-8184-e0630fc3a757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:00.462422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46729547-0981-4dab-8184-e0630fc3a757 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:00.472332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1130/2224] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:37:00.473214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46729547-0981-4dab-8184-e0630fc3a757 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:00.474541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-46729547-0981-4dab-8184-e0630fc3a757 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:00.534088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46729547-0981-4dab-8184-e0630fc3a757 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:00.538247 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e01732-a21c-4ae2-8756-4c98a8a62113 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:00.538891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e01732-a21c-4ae2-8756-4c98a8a62113 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:00.539174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e01732-a21c-4ae2-8756-4c98a8a62113 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:00.539451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e01732-a21c-4ae2-8756-4c98a8a62113 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:00.541236 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a4e01732-a21c-4ae2-8756-4c98a8a62113 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:00.563958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f27d2a76-a111-47d2-a82d-4deeca0c4c0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed240a780e2942eaafbc7b1aa33b1d0c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:00.607361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e01732-a21c-4ae2-8756-4c98a8a62113 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:00.608824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e01732-a21c-4ae2-8756-4c98a8a62113 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:00.618473 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1131/2225] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:37:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:37:00.669347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f27d2a76-a111-47d2-a82d-4deeca0c4c0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed240a780e2942eaafbc7b1aa33b1d0c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:00.671150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f27d2a76-a111-47d2-a82d-4deeca0c4c0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed240a780e2942eaafbc7b1aa33b1d0c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:00.695749 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f27d2a76-a111-47d2-a82d-4deeca0c4c0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed240a780e2942eaafbc7b1aa33b1d0c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:00.707464 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f27d2a76-a111-47d2-a82d-4deeca0c4c0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed240a780e2942eaafbc7b1aa33b1d0c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:00.747871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1095/2226] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:37:00 2022] DELETE /identity/v3/users/ed240a780e2942eaafbc7b1aa33b1d0c => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:00.762754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761aa8a-ffe3-4adc-b486-4c348a56237b None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:00.763144 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761aa8a-ffe3-4adc-b486-4c348a56237b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:00.763493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761aa8a-ffe3-4adc-b486-4c348a56237b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:00.783968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-d761aa8a-ffe3-4adc-b486-4c348a56237b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:00.785425 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d761aa8a-ffe3-4adc-b486-4c348a56237b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:00.850539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1132/2227] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:37:00.860357 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d2a76-a111-47d2-a82d-4deeca0c4c0b admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:00.861795 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f27d2a76-a111-47d2-a82d-4deeca0c4c0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:00.918103 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d2a76-a111-47d2-a82d-4deeca0c4c0b admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:00.921866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc4e4c-9f1b-4b12-88ce-0fe1aa6c3d1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:00.922475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc4e4c-9f1b-4b12-88ce-0fe1aa6c3d1b service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:00.922759 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc4e4c-9f1b-4b12-88ce-0fe1aa6c3d1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:00.923072 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc4e4c-9f1b-4b12-88ce-0fe1aa6c3d1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:00.932312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc4e4c-9f1b-4b12-88ce-0fe1aa6c3d1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:00.933761 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc4e4c-9f1b-4b12-88ce-0fe1aa6c3d1b service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:00.943771 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1096/2228] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:37:00.966110 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761aa8a-ffe3-4adc-b486-4c348a56237b None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:00.967821 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d761aa8a-ffe3-4adc-b486-4c348a56237b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:00.980117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761aa8a-ffe3-4adc-b486-4c348a56237b None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:00.984986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892362f-39ce-4f5a-8188-275b75c97a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:00.985631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892362f-39ce-4f5a-8188-275b75c97a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:00.985920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892362f-39ce-4f5a-8188-275b75c97a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:00.986335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892362f-39ce-4f5a-8188-275b75c97a7c admin admin] PATH_INFO: `/v3/projects/bbc4e17035aa4a1a9188f1177b92e089` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:00.987437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892362f-39ce-4f5a-8188-275b75c97a7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbc4e17035aa4a1a9188f1177b92e089)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:00.989407 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892362f-39ce-4f5a-8188-275b75c97a7c admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:01.046485 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:01.120699 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d892362f-39ce-4f5a-8188-275b75c97a7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbc4e17035aa4a1a9188f1177b92e089', 'request_id': 'req-d892362f-39ce-4f5a-8188-275b75c97a7c'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:01.127531 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d892362f-39ce-4f5a-8188-275b75c97a7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbc4e17035aa4a1a9188f1177b92e089', 'request_id': 'req-d892362f-39ce-4f5a-8188-275b75c97a7c'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:01.257307 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d892362f-39ce-4f5a-8188-275b75c97a7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbc4e17035aa4a1a9188f1177b92e089', 'request_id': 'req-d892362f-39ce-4f5a-8188-275b75c97a7c'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:01.268168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d892362f-39ce-4f5a-8188-275b75c97a7c admin admin] The token cache is being invalidate because project bbc4e17035aa4a1a9188f1177b92e089 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:37:01.269924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d892362f-39ce-4f5a-8188-275b75c97a7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:01.269924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d892362f-39ce-4f5a-8188-275b75c97a7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:01.270543 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1133/2229] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:37:00 2022] DELETE /identity/v3/projects/bbc4e17035aa4a1a9188f1177b92e089 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:01.274526 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc4e4c-9f1b-4b12-88ce-0fe1aa6c3d1b service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:01.284290 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc4e4c-9f1b-4b12-88ce-0fe1aa6c3d1b service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:01.343642 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ed341-0998-48e2-9b12-ec63a4d74a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:01.344403 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ed341-0998-48e2-9b12-ec63a4d74a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:01.344973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ed341-0998-48e2-9b12-ec63a4d74a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:01.345146 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ed341-0998-48e2-9b12-ec63a4d74a16 admin admin] PATH_INFO: `/v3/users/9b2f32643f0d410caca6a24c1446b982` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:01.346393 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ed341-0998-48e2-9b12-ec63a4d74a16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b2f32643f0d410caca6a24c1446b982)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:01.348575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ed341-0998-48e2-9b12-ec63a4d74a16 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:01.615826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-396ed341-0998-48e2-9b12-ec63a4d74a16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b2f32643f0d410caca6a24c1446b982'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:01.642254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892362f-39ce-4f5a-8188-275b75c97a7c admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:01.644731 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d892362f-39ce-4f5a-8188-275b75c97a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:01.660800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892362f-39ce-4f5a-8188-275b75c97a7c admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:01.667493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a06aee-3c91-499e-95fc-261647416a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:01.668051 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a06aee-3c91-499e-95fc-261647416a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:01.668334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a06aee-3c91-499e-95fc-261647416a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:01.668606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a06aee-3c91-499e-95fc-261647416a39 admin admin] PATH_INFO: `/v3/projects/e3c85a3787784f90a0814aed1762840f` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:01.669752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a06aee-3c91-499e-95fc-261647416a39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3c85a3787784f90a0814aed1762840f)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:01.672288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a06aee-3c91-499e-95fc-261647416a39 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:01.724853 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-396ed341-0998-48e2-9b12-ec63a4d74a16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b2f32643f0d410caca6a24c1446b982'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:01.725534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-396ed341-0998-48e2-9b12-ec63a4d74a16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b2f32643f0d410caca6a24c1446b982'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:01.738455 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-396ed341-0998-48e2-9b12-ec63a4d74a16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b2f32643f0d410caca6a24c1446b982'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:01.745148 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-396ed341-0998-48e2-9b12-ec63a4d74a16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b2f32643f0d410caca6a24c1446b982'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:01.751283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1097/2230] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:37:01 2022] DELETE /identity/v3/users/9b2f32643f0d410caca6a24c1446b982 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:01.891743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-49a06aee-3c91-499e-95fc-261647416a39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3c85a3787784f90a0814aed1762840f', 'request_id': 'req-49a06aee-3c91-499e-95fc-261647416a39'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:01.898051 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-49a06aee-3c91-499e-95fc-261647416a39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3c85a3787784f90a0814aed1762840f', 'request_id': 'req-49a06aee-3c91-499e-95fc-261647416a39'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:02.028194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-49a06aee-3c91-499e-95fc-261647416a39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3c85a3787784f90a0814aed1762840f', 'request_id': 'req-49a06aee-3c91-499e-95fc-261647416a39'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:02.040349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-49a06aee-3c91-499e-95fc-261647416a39 admin admin] The token cache is being invalidate because project e3c85a3787784f90a0814aed1762840f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:37:02.040646 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-49a06aee-3c91-499e-95fc-261647416a39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:02.045764 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-49a06aee-3c91-499e-95fc-261647416a39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:02.045764 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1134/2231] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:37:01 2022] DELETE /identity/v3/projects/e3c85a3787784f90a0814aed1762840f => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:02.047519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ed341-0998-48e2-9b12-ec63a4d74a16 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:02.049511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-396ed341-0998-48e2-9b12-ec63a4d74a16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:02.127332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ed341-0998-48e2-9b12-ec63a4d74a16 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:02.189000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f8d13-a700-4bfc-a6b5-3c50bc37d04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:02.189645 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f8d13-a700-4bfc-a6b5-3c50bc37d04d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:02.189964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f8d13-a700-4bfc-a6b5-3c50bc37d04d admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:02.190244 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f8d13-a700-4bfc-a6b5-3c50bc37d04d admin admin] PATH_INFO: `/v3/users/44bdf7e64f034961addb9f1a34ae0a34` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:02.191394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f8d13-a700-4bfc-a6b5-3c50bc37d04d admin admin] RBAC: Authorizing `identity:delete_user(user_id=44bdf7e64f034961addb9f1a34ae0a34)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:02.193485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f8d13-a700-4bfc-a6b5-3c50bc37d04d admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:02.333995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a06aee-3c91-499e-95fc-261647416a39 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:02.335634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-49a06aee-3c91-499e-95fc-261647416a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:02.348867 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a06aee-3c91-499e-95fc-261647416a39 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:02.353882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f034025-1ea2-4a73-8e31-fdce8c8c7bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:02.355580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f034025-1ea2-4a73-8e31-fdce8c8c7bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:02.355580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f034025-1ea2-4a73-8e31-fdce8c8c7bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:02.355580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f034025-1ea2-4a73-8e31-fdce8c8c7bf4 admin admin] PATH_INFO: `/v3/projects/8141b2cdbd0e4cd186b8e22b2786a673` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:02.357296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f034025-1ea2-4a73-8e31-fdce8c8c7bf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8141b2cdbd0e4cd186b8e22b2786a673)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:02.359219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f034025-1ea2-4a73-8e31-fdce8c8c7bf4 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:02.413301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e60f8d13-a700-4bfc-a6b5-3c50bc37d04d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44bdf7e64f034961addb9f1a34ae0a34'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:02.457345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0f034025-1ea2-4a73-8e31-fdce8c8c7bf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8141b2cdbd0e4cd186b8e22b2786a673', 'request_id': 'req-0f034025-1ea2-4a73-8e31-fdce8c8c7bf4'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:02.463964 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0f034025-1ea2-4a73-8e31-fdce8c8c7bf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8141b2cdbd0e4cd186b8e22b2786a673', 'request_id': 'req-0f034025-1ea2-4a73-8e31-fdce8c8c7bf4'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:02.468848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e60f8d13-a700-4bfc-a6b5-3c50bc37d04d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44bdf7e64f034961addb9f1a34ae0a34'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:02.469536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e60f8d13-a700-4bfc-a6b5-3c50bc37d04d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44bdf7e64f034961addb9f1a34ae0a34'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:02.483907 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e60f8d13-a700-4bfc-a6b5-3c50bc37d04d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44bdf7e64f034961addb9f1a34ae0a34'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:02.490864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e60f8d13-a700-4bfc-a6b5-3c50bc37d04d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44bdf7e64f034961addb9f1a34ae0a34'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:02.497251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1098/2232] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:37:02 2022] DELETE /identity/v3/users/44bdf7e64f034961addb9f1a34ae0a34 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:02.611660 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0f034025-1ea2-4a73-8e31-fdce8c8c7bf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8141b2cdbd0e4cd186b8e22b2786a673', 'request_id': 'req-0f034025-1ea2-4a73-8e31-fdce8c8c7bf4'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:02.618940 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0f034025-1ea2-4a73-8e31-fdce8c8c7bf4 admin admin] The token cache is being invalidate because project 8141b2cdbd0e4cd186b8e22b2786a673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:37:02.619233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0f034025-1ea2-4a73-8e31-fdce8c8c7bf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:02.619884 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0f034025-1ea2-4a73-8e31-fdce8c8c7bf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:02.621638 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1135/2233] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:37:02 2022] DELETE /identity/v3/projects/8141b2cdbd0e4cd186b8e22b2786a673 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:37:02.626701 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d989e-f039-4753-b3b7-bdc35f4f566e None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:02.627192 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d989e-f039-4753-b3b7-bdc35f4f566e None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:02.627473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d989e-f039-4753-b3b7-bdc35f4f566e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:02.648186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-7e2d989e-f039-4753-b3b7-bdc35f4f566e None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:02.649416 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-7e2d989e-f039-4753-b3b7-bdc35f4f566e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:02.709762 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1099/2234] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:37:02.719496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f7a1d-baac-4f64-b0eb-19868cf381f5 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:02.719854 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f7a1d-baac-4f64-b0eb-19868cf381f5 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:02.720198 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f7a1d-baac-4f64-b0eb-19868cf381f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:02.739716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-5d2f7a1d-baac-4f64-b0eb-19868cf381f5 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:02.746887 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5d2f7a1d-baac-4f64-b0eb-19868cf381f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:02.751264 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1136/2235] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:37:02.755426 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d989e-f039-4753-b3b7-bdc35f4f566e None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:02.765820 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d989e-f039-4753-b3b7-bdc35f4f566e None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:02.770642 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9e12e-c583-48de-ac0d-84ec5f4b0982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:02.771319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9e12e-c583-48de-ac0d-84ec5f4b0982 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:02.771603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9e12e-c583-48de-ac0d-84ec5f4b0982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:02.771878 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9e12e-c583-48de-ac0d-84ec5f4b0982 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:02.772566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9e12e-c583-48de-ac0d-84ec5f4b0982 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:37:02.772875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9e12e-c583-48de-ac0d-84ec5f4b0982 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:02.774520 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9e12e-c583-48de-ac0d-84ec5f4b0982 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:02.790559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1100/2236] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:37:02.791782 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f7a1d-baac-4f64-b0eb-19868cf381f5 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:02.800924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f7a1d-baac-4f64-b0eb-19868cf381f5 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:02.805632 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cb4a9-c14a-4ee2-9491-eff5a77b08ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:02.806227 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cb4a9-c14a-4ee2-9491-eff5a77b08ba admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:02.806513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cb4a9-c14a-4ee2-9491-eff5a77b08ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:02.806803 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cb4a9-c14a-4ee2-9491-eff5a77b08ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:02.807463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401cb4a9-c14a-4ee2-9491-eff5a77b08ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:02.809355 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401cb4a9-c14a-4ee2-9491-eff5a77b08ba admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:02.812200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:02.910949 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1137/2237] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:37:02 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:37:02.915054 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9e12e-c583-48de-ac0d-84ec5f4b0982 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:02.925775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9e12e-c583-48de-ac0d-84ec5f4b0982 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:02.996966 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cb4a9-c14a-4ee2-9491-eff5a77b08ba admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:02.997368 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90811e8-2c69-49c9-b77b-cfb4ad7d4fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:02.998198 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90811e8-2c69-49c9-b77b-cfb4ad7d4fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:02.998569 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-401cb4a9-c14a-4ee2-9491-eff5a77b08ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:02.998689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90811e8-2c69-49c9-b77b-cfb4ad7d4fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:02.999061 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90811e8-2c69-49c9-b77b-cfb4ad7d4fe6 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:02.999884 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90811e8-2c69-49c9-b77b-cfb4ad7d4fe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:03.002268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90811e8-2c69-49c9-b77b-cfb4ad7d4fe6 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:03.015580 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cb4a9-c14a-4ee2-9491-eff5a77b08ba admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:03.027575 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be071a0-e495-4135-ac1a-2d0f061c2639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:03.029066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be071a0-e495-4135-ac1a-2d0f061c2639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:03.030160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be071a0-e495-4135-ac1a-2d0f061c2639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:03.030692 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be071a0-e495-4135-ac1a-2d0f061c2639 admin admin] PATH_INFO: `/v3/projects/b9a8fc9c0d8d47ef9afe102f352f48e6` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:03.032983 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be071a0-e495-4135-ac1a-2d0f061c2639 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9a8fc9c0d8d47ef9afe102f352f48e6)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:03.043795 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be071a0-e495-4135-ac1a-2d0f061c2639 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:03.148372 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1101/2238] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:37:02 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:37:03.153054 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90811e8-2c69-49c9-b77b-cfb4ad7d4fe6 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:03.165347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90811e8-2c69-49c9-b77b-cfb4ad7d4fe6 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:03.165924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8be071a0-e495-4135-ac1a-2d0f061c2639 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9a8fc9c0d8d47ef9afe102f352f48e6', 'request_id': 'req-8be071a0-e495-4135-ac1a-2d0f061c2639'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:03.173737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8be071a0-e495-4135-ac1a-2d0f061c2639 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9a8fc9c0d8d47ef9afe102f352f48e6', 'request_id': 'req-8be071a0-e495-4135-ac1a-2d0f061c2639'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:03.226416 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b9d16-d454-40b2-935c-e7d7b36af8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:03.227121 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b9d16-d454-40b2-935c-e7d7b36af8ed admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:03.227413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b9d16-d454-40b2-935c-e7d7b36af8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:03.227688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b9d16-d454-40b2-935c-e7d7b36af8ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:03.228364 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b9d16-d454-40b2-935c-e7d7b36af8ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:03.229968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b9d16-d454-40b2-935c-e7d7b36af8ed admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:03.240551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1102/2239] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:37:03.244362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b9d16-d454-40b2-935c-e7d7b36af8ed admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:03.254529 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b9d16-d454-40b2-935c-e7d7b36af8ed admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:03.259426 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6facebfc-6f6c-435c-ad86-d337086d897c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:03.260025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6facebfc-6f6c-435c-ad86-d337086d897c admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:03.260310 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6facebfc-6f6c-435c-ad86-d337086d897c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:03.260587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6facebfc-6f6c-435c-ad86-d337086d897c admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:03.261253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6facebfc-6f6c-435c-ad86-d337086d897c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:03.262978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6facebfc-6f6c-435c-ad86-d337086d897c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:03.274340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1103/2240] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:37:03.278257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6facebfc-6f6c-435c-ad86-d337086d897c admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:03.280224 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8be071a0-e495-4135-ac1a-2d0f061c2639 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9a8fc9c0d8d47ef9afe102f352f48e6', 'request_id': 'req-8be071a0-e495-4135-ac1a-2d0f061c2639'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:03.290281 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6facebfc-6f6c-435c-ad86-d337086d897c admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:03.295038 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc8539-c348-48d3-98b8-e100317dd9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:03.295684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8be071a0-e495-4135-ac1a-2d0f061c2639 admin admin] The token cache is being invalidate because project b9a8fc9c0d8d47ef9afe102f352f48e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:37:03.295824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc8539-c348-48d3-98b8-e100317dd9dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:03.295966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc8539-c348-48d3-98b8-e100317dd9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:03.296242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc8539-c348-48d3-98b8-e100317dd9dc admin admin] PATH_INFO: `/v3/projects/1382cd3884c343df83d0b2dffd483e77/users/7bebac1e3eac4f21a531843bdcd0e8d0/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:03.296349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8be071a0-e495-4135-ac1a-2d0f061c2639 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:03.297254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8be071a0-e495-4135-ac1a-2d0f061c2639 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:03.298987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1138/2241] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:37:02 2022] DELETE /identity/v3/projects/b9a8fc9c0d8d47ef9afe102f352f48e6 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:37:03.303405 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be071a0-e495-4135-ac1a-2d0f061c2639 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:03.304371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8be071a0-e495-4135-ac1a-2d0f061c2639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:03.315994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc8539-c348-48d3-98b8-e100317dd9dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1382cd3884c343df83d0b2dffd483e77, user_id=7bebac1e3eac4f21a531843bdcd0e8d0, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:03.318418 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc8539-c348-48d3-98b8-e100317dd9dc admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:03.320004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be071a0-e495-4135-ac1a-2d0f061c2639 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:03.325368 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164cf6a-bb42-4c09-a5ff-9d5ec9bd45be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:03.326005 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164cf6a-bb42-4c09-a5ff-9d5ec9bd45be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:03.326430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164cf6a-bb42-4c09-a5ff-9d5ec9bd45be admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:03.326916 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164cf6a-bb42-4c09-a5ff-9d5ec9bd45be admin admin] PATH_INFO: `/v3/users/5c9ecc4df15d40c386bd4c1e1066b046` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:03.328604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164cf6a-bb42-4c09-a5ff-9d5ec9bd45be admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c9ecc4df15d40c386bd4c1e1066b046)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:03.330765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164cf6a-bb42-4c09-a5ff-9d5ec9bd45be admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:03.407152 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1104/2242] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:37:03 2022] PUT /identity/v3/projects/1382cd3884c343df83d0b2dffd483e77/users/7bebac1e3eac4f21a531843bdcd0e8d0/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:37:03.417635 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356240a1-c6fd-4c00-a618-f7dd2feb6e19 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:03.417942 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356240a1-c6fd-4c00-a618-f7dd2feb6e19 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:03.418291 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356240a1-c6fd-4c00-a618-f7dd2feb6e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:03.450570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-356240a1-c6fd-4c00-a618-f7dd2feb6e19 None None] MFA Rules not processed for user `7bebac1e3eac4f21a531843bdcd0e8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:03.451824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-356240a1-c6fd-4c00-a618-f7dd2feb6e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:03.515956 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1105/2243] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:37:03.526624 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356240a1-c6fd-4c00-a618-f7dd2feb6e19 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:03.527991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-356240a1-c6fd-4c00-a618-f7dd2feb6e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:03.560098 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b164cf6a-bb42-4c09-a5ff-9d5ec9bd45be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c9ecc4df15d40c386bd4c1e1066b046'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:03.597161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356240a1-c6fd-4c00-a618-f7dd2feb6e19 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:03.601658 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754838a-0886-4ee6-b2b1-efc8e9648a65 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '735986478c354ed196c601f12f3ac377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:03.602273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754838a-0886-4ee6-b2b1-efc8e9648a65 service glance] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:03.602586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754838a-0886-4ee6-b2b1-efc8e9648a65 service glance] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:03.602890 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754838a-0886-4ee6-b2b1-efc8e9648a65 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:03.612358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754838a-0886-4ee6-b2b1-efc8e9648a65 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:03.614945 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754838a-0886-4ee6-b2b1-efc8e9648a65 service glance] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:03.625310 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1106/2244] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:37:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:37:03.663503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b164cf6a-bb42-4c09-a5ff-9d5ec9bd45be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c9ecc4df15d40c386bd4c1e1066b046'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:03.678368 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b164cf6a-bb42-4c09-a5ff-9d5ec9bd45be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c9ecc4df15d40c386bd4c1e1066b046'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:03.679194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b164cf6a-bb42-4c09-a5ff-9d5ec9bd45be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c9ecc4df15d40c386bd4c1e1066b046'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:03.721469 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b164cf6a-bb42-4c09-a5ff-9d5ec9bd45be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c9ecc4df15d40c386bd4c1e1066b046'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:03.732503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1139/2245] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:37:03 2022] DELETE /identity/v3/users/5c9ecc4df15d40c386bd4c1e1066b046 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:37:04.046210 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754838a-0886-4ee6-b2b1-efc8e9648a65 service glance] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:04.047700 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f754838a-0886-4ee6-b2b1-efc8e9648a65 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:04.121634 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:04.128237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754838a-0886-4ee6-b2b1-efc8e9648a65 service glance] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:04.185145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b223db-fb49-48c2-ba7d-5cb93f5651bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:04.185733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b223db-fb49-48c2-ba7d-5cb93f5651bc service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:04.186012 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b223db-fb49-48c2-ba7d-5cb93f5651bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:04.186312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b223db-fb49-48c2-ba7d-5cb93f5651bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:04.188102 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-28b223db-fb49-48c2-ba7d-5cb93f5651bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:04.247530 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b223db-fb49-48c2-ba7d-5cb93f5651bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:04.248978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b223db-fb49-48c2-ba7d-5cb93f5651bc service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:04.258391 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1107/2246] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:37:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:37:04.299565 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164cf6a-bb42-4c09-a5ff-9d5ec9bd45be admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:04.301181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b164cf6a-bb42-4c09-a5ff-9d5ec9bd45be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:04.403477 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164cf6a-bb42-4c09-a5ff-9d5ec9bd45be admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:04.408991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ed963-e411-4990-a9bb-1b8a5aeaa541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:04.412892 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ed963-e411-4990-a9bb-1b8a5aeaa541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:04.412892 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ed963-e411-4990-a9bb-1b8a5aeaa541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:04.412892 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ed963-e411-4990-a9bb-1b8a5aeaa541 admin admin] PATH_INFO: `/v3/projects/e5dab1c4dd4b4973ac2630ba9519f983` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:04.412892 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ed963-e411-4990-a9bb-1b8a5aeaa541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5dab1c4dd4b4973ac2630ba9519f983)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:04.413272 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ed963-e411-4990-a9bb-1b8a5aeaa541 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:04.571559 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-539ed963-e411-4990-a9bb-1b8a5aeaa541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5dab1c4dd4b4973ac2630ba9519f983', 'request_id': 'req-539ed963-e411-4990-a9bb-1b8a5aeaa541'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:04.577938 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-539ed963-e411-4990-a9bb-1b8a5aeaa541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5dab1c4dd4b4973ac2630ba9519f983', 'request_id': 'req-539ed963-e411-4990-a9bb-1b8a5aeaa541'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:04.680136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-539ed963-e411-4990-a9bb-1b8a5aeaa541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5dab1c4dd4b4973ac2630ba9519f983', 'request_id': 'req-539ed963-e411-4990-a9bb-1b8a5aeaa541'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:04.691535 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-539ed963-e411-4990-a9bb-1b8a5aeaa541 admin admin] The token cache is being invalidate because project e5dab1c4dd4b4973ac2630ba9519f983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:37:04.691890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-539ed963-e411-4990-a9bb-1b8a5aeaa541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:04.692533 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-539ed963-e411-4990-a9bb-1b8a5aeaa541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:04.694079 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1140/2247] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:37:04 2022] DELETE /identity/v3/projects/e5dab1c4dd4b4973ac2630ba9519f983 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:04.700073 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978a12e-9b82-4fee-94bb-b5e6195b68c3 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:04.700429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978a12e-9b82-4fee-94bb-b5e6195b68c3 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:04.700707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978a12e-9b82-4fee-94bb-b5e6195b68c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:04.721034 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:04.730096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-a978a12e-9b82-4fee-94bb-b5e6195b68c3 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:04.730096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a978a12e-9b82-4fee-94bb-b5e6195b68c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:04.792341 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1108/2248] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:37:04.801067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378624f-19b4-4a2b-97b7-7aac996b1418 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:04.801431 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378624f-19b4-4a2b-97b7-7aac996b1418 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:04.801791 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378624f-19b4-4a2b-97b7-7aac996b1418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:04.822265 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-0378624f-19b4-4a2b-97b7-7aac996b1418 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:04.823601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0378624f-19b4-4a2b-97b7-7aac996b1418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:04.831589 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1141/2249] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:37:04.835828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978a12e-9b82-4fee-94bb-b5e6195b68c3 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:04.845713 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978a12e-9b82-4fee-94bb-b5e6195b68c3 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:04.850718 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5c30a-35c5-4459-b0fe-3a00577927c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:04.851350 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5c30a-35c5-4459-b0fe-3a00577927c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:04.851642 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5c30a-35c5-4459-b0fe-3a00577927c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:04.851917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5c30a-35c5-4459-b0fe-3a00577927c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:04.852565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5c30a-35c5-4459-b0fe-3a00577927c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:37:04.852872 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5c30a-35c5-4459-b0fe-3a00577927c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:04.854511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5c30a-35c5-4459-b0fe-3a00577927c1 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:04.870375 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1109/2250] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:37:04.871692 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378624f-19b4-4a2b-97b7-7aac996b1418 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:04.881797 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378624f-19b4-4a2b-97b7-7aac996b1418 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:04.886562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3d831-f23f-4b6f-ac36-02212c19c05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:04.887198 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3d831-f23f-4b6f-ac36-02212c19c05a admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:04.887474 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3d831-f23f-4b6f-ac36-02212c19c05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:04.887743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3d831-f23f-4b6f-ac36-02212c19c05a admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:04.888403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3d831-f23f-4b6f-ac36-02212c19c05a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:04.891204 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3d831-f23f-4b6f-ac36-02212c19c05a admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:04.990829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1142/2251] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:37:04 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:37:04.994625 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5c30a-35c5-4459-b0fe-3a00577927c1 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:05.006315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5c30a-35c5-4459-b0fe-3a00577927c1 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:05.071862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c02bd-8171-410d-a986-d836fa856639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:05.072498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c02bd-8171-410d-a986-d836fa856639 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:05.072793 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c02bd-8171-410d-a986-d836fa856639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:05.073204 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c02bd-8171-410d-a986-d836fa856639 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:05.073826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c02bd-8171-410d-a986-d836fa856639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:05.078814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c02bd-8171-410d-a986-d836fa856639 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:05.266804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1110/2252] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:37:04 2022] POST /identity/v3/users => generated 283 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:37:05.270750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3d831-f23f-4b6f-ac36-02212c19c05a admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:05.281246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3d831-f23f-4b6f-ac36-02212c19c05a admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:05.285927 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70225188-4e49-469e-abe0-150431b5850f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:05.286530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70225188-4e49-469e-abe0-150431b5850f admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:05.286837 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70225188-4e49-469e-abe0-150431b5850f admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:05.287118 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70225188-4e49-469e-abe0-150431b5850f admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:05.287763 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70225188-4e49-469e-abe0-150431b5850f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:05.289368 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70225188-4e49-469e-abe0-150431b5850f admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:05.299647 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1143/2253] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:37:05.303625 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c02bd-8171-410d-a986-d836fa856639 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:05.314598 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c02bd-8171-410d-a986-d836fa856639 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:05.319773 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fa557-60d6-4668-a745-8899d90321b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:05.320387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fa557-60d6-4668-a745-8899d90321b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:05.321947 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fa557-60d6-4668-a745-8899d90321b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:05.321947 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fa557-60d6-4668-a745-8899d90321b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:05.321947 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995fa557-60d6-4668-a745-8899d90321b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:05.323217 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995fa557-60d6-4668-a745-8899d90321b9 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:05.334675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1111/2254] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:37:05.338850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70225188-4e49-469e-abe0-150431b5850f admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:05.349708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70225188-4e49-469e-abe0-150431b5850f admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:05.354728 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53a97f-869c-488a-93a0-c720205fb2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:05.355223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53a97f-869c-488a-93a0-c720205fb2b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:05.355521 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53a97f-869c-488a-93a0-c720205fb2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:05.355786 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53a97f-869c-488a-93a0-c720205fb2b0 admin admin] PATH_INFO: `/v3/projects/9a90ef08cb244ff8b591198302887262/users/5e60199963614e8fa718c15de15a9512/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:05.373893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:05.381598 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53a97f-869c-488a-93a0-c720205fb2b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a90ef08cb244ff8b591198302887262, user_id=5e60199963614e8fa718c15de15a9512, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:05.384807 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53a97f-869c-488a-93a0-c720205fb2b0 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:05.459047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1144/2255] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:37:05 2022] PUT /identity/v3/projects/9a90ef08cb244ff8b591198302887262/users/5e60199963614e8fa718c15de15a9512/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:05.474447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6020f99-6d91-46ae-ab9c-8e637ceb5f29 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:05.474737 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6020f99-6d91-46ae-ab9c-8e637ceb5f29 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:05.475108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6020f99-6d91-46ae-ab9c-8e637ceb5f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:05.532496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-a6020f99-6d91-46ae-ab9c-8e637ceb5f29 None None] MFA Rules not processed for user `5e60199963614e8fa718c15de15a9512`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:05.533790 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a6020f99-6d91-46ae-ab9c-8e637ceb5f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:05.638831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1112/2256] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:37:05.644830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53a97f-869c-488a-93a0-c720205fb2b0 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:05.753348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53a97f-869c-488a-93a0-c720205fb2b0 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:05.759157 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cfac9-bec0-4e98-8e12-34dc1c56e23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:05.759784 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cfac9-bec0-4e98-8e12-34dc1c56e23f admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:05.760065 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cfac9-bec0-4e98-8e12-34dc1c56e23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:05.760338 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cfac9-bec0-4e98-8e12-34dc1c56e23f admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:05.761015 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092cfac9-bec0-4e98-8e12-34dc1c56e23f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:05.762965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092cfac9-bec0-4e98-8e12-34dc1c56e23f admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:05.912445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1145/2257] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:37:05 2022] POST /identity/v3/projects => generated 367 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:37:05.916300 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6020f99-6d91-46ae-ab9c-8e637ceb5f29 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:05.928490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6020f99-6d91-46ae-ab9c-8e637ceb5f29 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:06.009990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e878c0-c3d0-4c14-9952-e2aa72b99856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:06.010531 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e878c0-c3d0-4c14-9952-e2aa72b99856 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:06.010844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e878c0-c3d0-4c14-9952-e2aa72b99856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:06.011132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e878c0-c3d0-4c14-9952-e2aa72b99856 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:06.011836 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e878c0-c3d0-4c14-9952-e2aa72b99856 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:06.013866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e878c0-c3d0-4c14-9952-e2aa72b99856 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:06.120856 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1113/2258] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:37:05 2022] POST /identity/v3/users => generated 290 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:37:06.124906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cfac9-bec0-4e98-8e12-34dc1c56e23f admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:06.135437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cfac9-bec0-4e98-8e12-34dc1c56e23f admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:06.142540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b4ebc-4de0-4334-a9f1-6ac36c8de738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:06.143185 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b4ebc-4de0-4334-a9f1-6ac36c8de738 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:06.143462 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b4ebc-4de0-4334-a9f1-6ac36c8de738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:06.143732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b4ebc-4de0-4334-a9f1-6ac36c8de738 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:06.144415 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b4ebc-4de0-4334-a9f1-6ac36c8de738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:06.145972 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b4ebc-4de0-4334-a9f1-6ac36c8de738 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:06.157808 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1146/2259] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:37:06.161969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e878c0-c3d0-4c14-9952-e2aa72b99856 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:06.172819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e878c0-c3d0-4c14-9952-e2aa72b99856 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:06.183810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492b84d-0fb4-48ed-8db4-7c2c19336b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:06.184490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492b84d-0fb4-48ed-8db4-7c2c19336b3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:06.184777 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492b84d-0fb4-48ed-8db4-7c2c19336b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:06.185058 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492b84d-0fb4-48ed-8db4-7c2c19336b3a admin admin] PATH_INFO: `/v3/projects/6bf138ab196043f9bacc2cfb15e8dc44/users/44a68887d49449e4a4b3911d14a03ed6/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:06.201888 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9492b84d-0fb4-48ed-8db4-7c2c19336b3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bf138ab196043f9bacc2cfb15e8dc44, user_id=44a68887d49449e4a4b3911d14a03ed6, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:06.205920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9492b84d-0fb4-48ed-8db4-7c2c19336b3a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:06.275523 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b4ebc-4de0-4334-a9f1-6ac36c8de738 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:06.277292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-1d9b4ebc-4de0-4334-a9f1-6ac36c8de738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:06.338407 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1114/2260] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:37:06 2022] PUT /identity/v3/projects/6bf138ab196043f9bacc2cfb15e8dc44/users/44a68887d49449e4a4b3911d14a03ed6/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:06.356311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b4ebc-4de0-4334-a9f1-6ac36c8de738 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:06.358602 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492b84d-0fb4-48ed-8db4-7c2c19336b3a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:06.415359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929760a-bada-4bde-9d39-06405b32e761 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:06.416112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929760a-bada-4bde-9d39-06405b32e761 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:06.416397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929760a-bada-4bde-9d39-06405b32e761 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:06.416696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929760a-bada-4bde-9d39-06405b32e761 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:06.418732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b929760a-bada-4bde-9d39-06405b32e761 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:06.437068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492b84d-0fb4-48ed-8db4-7c2c19336b3a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:06.442434 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af113b-7d47-4230-94b5-c19e830877ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:06.443065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af113b-7d47-4230-94b5-c19e830877ad admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:06.443350 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af113b-7d47-4230-94b5-c19e830877ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:06.443743 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af113b-7d47-4230-94b5-c19e830877ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:06.444352 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af113b-7d47-4230-94b5-c19e830877ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:06.445953 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af113b-7d47-4230-94b5-c19e830877ad admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:06.457951 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1115/2261] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:37:06.462090 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af113b-7d47-4230-94b5-c19e830877ad admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:06.472062 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af113b-7d47-4230-94b5-c19e830877ad admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:06.476769 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a188f-d65e-45ab-8752-eba148bdfe83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:06.477408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a188f-d65e-45ab-8752-eba148bdfe83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:06.477690 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a188f-d65e-45ab-8752-eba148bdfe83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:06.477964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a188f-d65e-45ab-8752-eba148bdfe83 admin admin] PATH_INFO: `/v3/projects/6bf138ab196043f9bacc2cfb15e8dc44/users/44a68887d49449e4a4b3911d14a03ed6/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:06.479906 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a188f-d65e-45ab-8752-eba148bdfe83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bf138ab196043f9bacc2cfb15e8dc44, user_id=44a68887d49449e4a4b3911d14a03ed6, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:06.482554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a188f-d65e-45ab-8752-eba148bdfe83 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:06.484650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b929760a-bada-4bde-9d39-06405b32e761 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:06.486112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b929760a-bada-4bde-9d39-06405b32e761 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:06.493392 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1116/2262] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:37:06 2022] PUT /identity/v3/projects/6bf138ab196043f9bacc2cfb15e8dc44/users/44a68887d49449e4a4b3911d14a03ed6/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:06.496718 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1147/2263] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:37:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:37:06.504356 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e7a66-84ab-4d68-9a6c-7e898f72aeae None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:06.504703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e7a66-84ab-4d68-9a6c-7e898f72aeae None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:06.505005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e7a66-84ab-4d68-9a6c-7e898f72aeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:06.538098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-572e7a66-84ab-4d68-9a6c-7e898f72aeae None None] MFA Rules not processed for user `44a68887d49449e4a4b3911d14a03ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:06.539370 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-572e7a66-84ab-4d68-9a6c-7e898f72aeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:06.605947 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1117/2264] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:06 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:37:06.618102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929760a-bada-4bde-9d39-06405b32e761 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:06.619591 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b929760a-bada-4bde-9d39-06405b32e761 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:06.631101 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929760a-bada-4bde-9d39-06405b32e761 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:06.684513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923d5c3-2494-4ad5-948d-f85ea1a24e80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:06.685183 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923d5c3-2494-4ad5-948d-f85ea1a24e80 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:06.685405 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923d5c3-2494-4ad5-948d-f85ea1a24e80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:06.685681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923d5c3-2494-4ad5-948d-f85ea1a24e80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:06.699626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2923d5c3-2494-4ad5-948d-f85ea1a24e80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:06.701248 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2923d5c3-2494-4ad5-948d-f85ea1a24e80 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:06.712130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1148/2265] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:37:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:37:06.980291 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e7a66-84ab-4d68-9a6c-7e898f72aeae None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:06.990037 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e7a66-84ab-4d68-9a6c-7e898f72aeae None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:06.993641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ef2c6-daf7-4032-be56-a22f60e6c161 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:06.994243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ef2c6-daf7-4032-be56-a22f60e6c161 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:06.994526 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ef2c6-daf7-4032-be56-a22f60e6c161 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:06.994816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ef2c6-daf7-4032-be56-a22f60e6c161 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:07.051688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ef2c6-daf7-4032-be56-a22f60e6c161 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:07.053113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ef2c6-daf7-4032-be56-a22f60e6c161 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:07.061764 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1118/2266] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:37:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:37:07.549323 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:07.903322 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:09.200733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:09.325929 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:10.440513 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:10.771690 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:11.577577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:11.972086 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:12.829322 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923d5c3-2494-4ad5-948d-f85ea1a24e80 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:12.830959 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-2923d5c3-2494-4ad5-948d-f85ea1a24e80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:12.903774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923d5c3-2494-4ad5-948d-f85ea1a24e80 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:12.909250 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53326f3-0a3d-449c-a12b-1859b79ec320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:12.909889 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53326f3-0a3d-449c-a12b-1859b79ec320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:12.910168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53326f3-0a3d-449c-a12b-1859b79ec320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:12.910438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53326f3-0a3d-449c-a12b-1859b79ec320 admin admin] PATH_INFO: `/v3/users/675a42d11a1a4931a36bf223633d5a04` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:12.911609 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53326f3-0a3d-449c-a12b-1859b79ec320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=675a42d11a1a4931a36bf223633d5a04)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:12.913642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53326f3-0a3d-449c-a12b-1859b79ec320 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:13.232143 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f53326f3-0a3d-449c-a12b-1859b79ec320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '675a42d11a1a4931a36bf223633d5a04'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:13.353364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f53326f3-0a3d-449c-a12b-1859b79ec320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '675a42d11a1a4931a36bf223633d5a04'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:13.369006 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f53326f3-0a3d-449c-a12b-1859b79ec320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '675a42d11a1a4931a36bf223633d5a04'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:13.369846 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f53326f3-0a3d-449c-a12b-1859b79ec320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '675a42d11a1a4931a36bf223633d5a04'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:13.377862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f53326f3-0a3d-449c-a12b-1859b79ec320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '675a42d11a1a4931a36bf223633d5a04'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:13.388825 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1149/2267] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:37:12 2022] DELETE /identity/v3/users/675a42d11a1a4931a36bf223633d5a04 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:13.388825 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:13.390819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bac092-0a79-4104-b1c1-c60c9eb3fa7b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:13.391307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bac092-0a79-4104-b1c1-c60c9eb3fa7b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:13.391517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bac092-0a79-4104-b1c1-c60c9eb3fa7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:13.415341 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-d9bac092-0a79-4104-b1c1-c60c9eb3fa7b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:13.421703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d9bac092-0a79-4104-b1c1-c60c9eb3fa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:13.494087 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1119/2268] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:37:13.507037 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53326f3-0a3d-449c-a12b-1859b79ec320 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:13.507037 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f53326f3-0a3d-449c-a12b-1859b79ec320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:13.604113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53326f3-0a3d-449c-a12b-1859b79ec320 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:13.607858 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a58a25-2d65-4cb5-a486-d57e9945ba1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:13.608465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a58a25-2d65-4cb5-a486-d57e9945ba1d service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:13.608745 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a58a25-2d65-4cb5-a486-d57e9945ba1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:13.609077 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a58a25-2d65-4cb5-a486-d57e9945ba1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:13.618244 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a58a25-2d65-4cb5-a486-d57e9945ba1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:13.619752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a58a25-2d65-4cb5-a486-d57e9945ba1d service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:13.629673 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1150/2269] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:37:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:37:13.803977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:14.401856 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bac092-0a79-4104-b1c1-c60c9eb3fa7b None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:14.403931 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d9bac092-0a79-4104-b1c1-c60c9eb3fa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:14.419251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bac092-0a79-4104-b1c1-c60c9eb3fa7b None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:14.426474 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7713e1-f007-4763-a9f7-75f740dd74f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:14.427366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7713e1-f007-4763-a9f7-75f740dd74f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:14.431253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7713e1-f007-4763-a9f7-75f740dd74f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:14.431253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7713e1-f007-4763-a9f7-75f740dd74f2 admin admin] PATH_INFO: `/v3/projects/1ae8c0838b56416e97b99fb71c53fe5b` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:14.431253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7713e1-f007-4763-a9f7-75f740dd74f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae8c0838b56416e97b99fb71c53fe5b)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:14.431253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7713e1-f007-4763-a9f7-75f740dd74f2 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:14.595579 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1e7713e1-f007-4763-a9f7-75f740dd74f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae8c0838b56416e97b99fb71c53fe5b', 'request_id': 'req-1e7713e1-f007-4763-a9f7-75f740dd74f2'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:14.604884 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1e7713e1-f007-4763-a9f7-75f740dd74f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae8c0838b56416e97b99fb71c53fe5b', 'request_id': 'req-1e7713e1-f007-4763-a9f7-75f740dd74f2'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:14.777693 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1e7713e1-f007-4763-a9f7-75f740dd74f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae8c0838b56416e97b99fb71c53fe5b', 'request_id': 'req-1e7713e1-f007-4763-a9f7-75f740dd74f2'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:14.794127 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1e7713e1-f007-4763-a9f7-75f740dd74f2 admin admin] The token cache is being invalidate because project 1ae8c0838b56416e97b99fb71c53fe5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:37:14.828949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1e7713e1-f007-4763-a9f7-75f740dd74f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:14.828949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1e7713e1-f007-4763-a9f7-75f740dd74f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:14.828949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1120/2270] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:37:14 2022] DELETE /identity/v3/projects/1ae8c0838b56416e97b99fb71c53fe5b => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:14.829439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a58a25-2d65-4cb5-a486-d57e9945ba1d service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:14.829439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a58a25-2d65-4cb5-a486-d57e9945ba1d service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:14.915354 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52a8e6-7ea4-4084-903a-db069a18f7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:14.916331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52a8e6-7ea4-4084-903a-db069a18f7dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:14.916730 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52a8e6-7ea4-4084-903a-db069a18f7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:14.917068 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52a8e6-7ea4-4084-903a-db069a18f7dc admin admin] PATH_INFO: `/v3/users/1e6d86d2b0854fb99168b7b1879a1b79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:14.918301 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52a8e6-7ea4-4084-903a-db069a18f7dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e6d86d2b0854fb99168b7b1879a1b79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:14.920443 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52a8e6-7ea4-4084-903a-db069a18f7dc admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:14.972481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:15.189933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-3a52a8e6-7ea4-4084-903a-db069a18f7dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e6d86d2b0854fb99168b7b1879a1b79'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:15.320928 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-3a52a8e6-7ea4-4084-903a-db069a18f7dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e6d86d2b0854fb99168b7b1879a1b79'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:15.334285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-3a52a8e6-7ea4-4084-903a-db069a18f7dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e6d86d2b0854fb99168b7b1879a1b79'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:15.335345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-3a52a8e6-7ea4-4084-903a-db069a18f7dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e6d86d2b0854fb99168b7b1879a1b79'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:15.344485 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-3a52a8e6-7ea4-4084-903a-db069a18f7dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e6d86d2b0854fb99168b7b1879a1b79'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:15.351492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1151/2271] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:37:14 2022] DELETE /identity/v3/users/1e6d86d2b0854fb99168b7b1879a1b79 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:15.726329 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7713e1-f007-4763-a9f7-75f740dd74f2 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:15.727991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-1e7713e1-f007-4763-a9f7-75f740dd74f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:15.756378 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7713e1-f007-4763-a9f7-75f740dd74f2 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:15.865088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f847907-b35a-4bb2-8dd1-dbf96a3a2bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:15.865088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f847907-b35a-4bb2-8dd1-dbf96a3a2bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:15.865486 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f847907-b35a-4bb2-8dd1-dbf96a3a2bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:15.865587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f847907-b35a-4bb2-8dd1-dbf96a3a2bd2 admin admin] PATH_INFO: `/v3/users/5e60199963614e8fa718c15de15a9512` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:15.866920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f847907-b35a-4bb2-8dd1-dbf96a3a2bd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e60199963614e8fa718c15de15a9512)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:15.869000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f847907-b35a-4bb2-8dd1-dbf96a3a2bd2 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:16.142664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52a8e6-7ea4-4084-903a-db069a18f7dc admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:16.144535 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-3a52a8e6-7ea4-4084-903a-db069a18f7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:16.165167 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52a8e6-7ea4-4084-903a-db069a18f7dc admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:16.180695 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f48d0-d569-4231-81c7-9ff9d0f6eab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:16.182020 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f48d0-d569-4231-81c7-9ff9d0f6eab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:16.182401 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f48d0-d569-4231-81c7-9ff9d0f6eab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:16.192749 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f48d0-d569-4231-81c7-9ff9d0f6eab7 admin admin] PATH_INFO: `/v3/projects/5aaa593cc8f046dabf2284143438326b` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:16.197436 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f48d0-d569-4231-81c7-9ff9d0f6eab7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aaa593cc8f046dabf2284143438326b)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:16.200119 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f48d0-d569-4231-81c7-9ff9d0f6eab7 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:16.287889 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3f847907-b35a-4bb2-8dd1-dbf96a3a2bd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e60199963614e8fa718c15de15a9512'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:16.339050 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-633f48d0-d569-4231-81c7-9ff9d0f6eab7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aaa593cc8f046dabf2284143438326b', 'request_id': 'req-633f48d0-d569-4231-81c7-9ff9d0f6eab7'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:16.346043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-633f48d0-d569-4231-81c7-9ff9d0f6eab7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aaa593cc8f046dabf2284143438326b', 'request_id': 'req-633f48d0-d569-4231-81c7-9ff9d0f6eab7'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:16.377063 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3f847907-b35a-4bb2-8dd1-dbf96a3a2bd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e60199963614e8fa718c15de15a9512'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:16.377643 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3f847907-b35a-4bb2-8dd1-dbf96a3a2bd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e60199963614e8fa718c15de15a9512'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:16.391958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3f847907-b35a-4bb2-8dd1-dbf96a3a2bd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e60199963614e8fa718c15de15a9512'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:16.399726 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3f847907-b35a-4bb2-8dd1-dbf96a3a2bd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e60199963614e8fa718c15de15a9512'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:16.407168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1121/2272] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:37:15 2022] DELETE /identity/v3/users/5e60199963614e8fa718c15de15a9512 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:16.411545 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252bffb-0ffa-4556-8e6c-56193544a9fc None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:16.411833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252bffb-0ffa-4556-8e6c-56193544a9fc None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:16.412111 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252bffb-0ffa-4556-8e6c-56193544a9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:16.434530 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-f252bffb-0ffa-4556-8e6c-56193544a9fc None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:16.435899 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f252bffb-0ffa-4556-8e6c-56193544a9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:16.458064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-633f48d0-d569-4231-81c7-9ff9d0f6eab7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aaa593cc8f046dabf2284143438326b', 'request_id': 'req-633f48d0-d569-4231-81c7-9ff9d0f6eab7'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:16.471985 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-633f48d0-d569-4231-81c7-9ff9d0f6eab7 admin admin] The token cache is being invalidate because project 5aaa593cc8f046dabf2284143438326b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:37:16.472498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-633f48d0-d569-4231-81c7-9ff9d0f6eab7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:16.473742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-633f48d0-d569-4231-81c7-9ff9d0f6eab7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:16.475361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1152/2273] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:37:16 2022] DELETE /identity/v3/projects/5aaa593cc8f046dabf2284143438326b => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:16.482739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f48d0-d569-4231-81c7-9ff9d0f6eab7 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:16.484988 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-633f48d0-d569-4231-81c7-9ff9d0f6eab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:16.506901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1122/2274] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:37:16.519277 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252bffb-0ffa-4556-8e6c-56193544a9fc None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:16.520637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f252bffb-0ffa-4556-8e6c-56193544a9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:16.550458 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f48d0-d569-4231-81c7-9ff9d0f6eab7 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:16.555180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f48afb-de40-4a50-8e4c-b417a627eee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:16.555847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f48afb-de40-4a50-8e4c-b417a627eee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:16.556129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f48afb-de40-4a50-8e4c-b417a627eee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:16.556402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f48afb-de40-4a50-8e4c-b417a627eee2 admin admin] PATH_INFO: `/v3/users/524576446f8e46eeaf373a32d03db77b` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:16.557536 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f48afb-de40-4a50-8e4c-b417a627eee2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=524576446f8e46eeaf373a32d03db77b)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:16.559522 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f48afb-de40-4a50-8e4c-b417a627eee2 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:16.577011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252bffb-0ffa-4556-8e6c-56193544a9fc None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:16.580773 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b11b54-02b1-4eef-91aa-cf9ac2fe739e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:16.581551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b11b54-02b1-4eef-91aa-cf9ac2fe739e service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:16.581796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b11b54-02b1-4eef-91aa-cf9ac2fe739e service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:16.582132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b11b54-02b1-4eef-91aa-cf9ac2fe739e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:16.591490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b11b54-02b1-4eef-91aa-cf9ac2fe739e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:16.592927 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b11b54-02b1-4eef-91aa-cf9ac2fe739e service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:16.603415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1123/2275] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:37:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:37:16.769057 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-88f48afb-de40-4a50-8e4c-b417a627eee2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '524576446f8e46eeaf373a32d03db77b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:16.885946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-88f48afb-de40-4a50-8e4c-b417a627eee2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '524576446f8e46eeaf373a32d03db77b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:16.900586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-88f48afb-de40-4a50-8e4c-b417a627eee2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '524576446f8e46eeaf373a32d03db77b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:16.901550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-88f48afb-de40-4a50-8e4c-b417a627eee2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '524576446f8e46eeaf373a32d03db77b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:16.909478 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-88f48afb-de40-4a50-8e4c-b417a627eee2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '524576446f8e46eeaf373a32d03db77b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:16.915771 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1153/2276] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:37:16 2022] DELETE /identity/v3/users/524576446f8e46eeaf373a32d03db77b => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:17.034835 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:17.147949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b11b54-02b1-4eef-91aa-cf9ac2fe739e service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:17.147949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-05b11b54-02b1-4eef-91aa-cf9ac2fe739e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:17.160438 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b11b54-02b1-4eef-91aa-cf9ac2fe739e service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:17.227904 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45d192-5472-45f1-a644-99b7d3a3860f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:17.228554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45d192-5472-45f1-a644-99b7d3a3860f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:17.228838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45d192-5472-45f1-a644-99b7d3a3860f admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:17.229121 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45d192-5472-45f1-a644-99b7d3a3860f admin admin] PATH_INFO: `/v3/projects/9a90ef08cb244ff8b591198302887262` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:17.230278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45d192-5472-45f1-a644-99b7d3a3860f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a90ef08cb244ff8b591198302887262)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:17.232511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45d192-5472-45f1-a644-99b7d3a3860f admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:17.393109 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9c45d192-5472-45f1-a644-99b7d3a3860f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a90ef08cb244ff8b591198302887262', 'request_id': 'req-9c45d192-5472-45f1-a644-99b7d3a3860f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:17.398361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9c45d192-5472-45f1-a644-99b7d3a3860f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a90ef08cb244ff8b591198302887262', 'request_id': 'req-9c45d192-5472-45f1-a644-99b7d3a3860f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:17.414295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f48afb-de40-4a50-8e4c-b417a627eee2 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:17.416442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-88f48afb-de40-4a50-8e4c-b417a627eee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:17.434847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f48afb-de40-4a50-8e4c-b417a627eee2 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:17.492056 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:17.492689 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:17.492968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:17.493240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2 admin admin] PATH_INFO: `/v3/projects/b252456d95d94021804c380e4b537113` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:17.494349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b252456d95d94021804c380e4b537113)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:17.496274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:17.508564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9c45d192-5472-45f1-a644-99b7d3a3860f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a90ef08cb244ff8b591198302887262', 'request_id': 'req-9c45d192-5472-45f1-a644-99b7d3a3860f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:17.523588 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9c45d192-5472-45f1-a644-99b7d3a3860f admin admin] The token cache is being invalidate because project 9a90ef08cb244ff8b591198302887262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:37:17.523886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9c45d192-5472-45f1-a644-99b7d3a3860f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:17.526708 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9c45d192-5472-45f1-a644-99b7d3a3860f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:17.526708 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1124/2277] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:37:17 2022] DELETE /identity/v3/projects/9a90ef08cb244ff8b591198302887262 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:17.530161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45d192-5472-45f1-a644-99b7d3a3860f admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:17.532456 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9c45d192-5472-45f1-a644-99b7d3a3860f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:17.545386 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45d192-5472-45f1-a644-99b7d3a3860f admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:17.550285 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d5976-d39a-4334-9cbc-6f4e9165a2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:17.550945 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d5976-d39a-4334-9cbc-6f4e9165a2d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:17.551231 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d5976-d39a-4334-9cbc-6f4e9165a2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:17.551508 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d5976-d39a-4334-9cbc-6f4e9165a2d6 admin admin] PATH_INFO: `/v3/users/44a68887d49449e4a4b3911d14a03ed6` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:17.552667 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d5976-d39a-4334-9cbc-6f4e9165a2d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44a68887d49449e4a4b3911d14a03ed6)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:17.554635 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d5976-d39a-4334-9cbc-6f4e9165a2d6 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:17.652448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b252456d95d94021804c380e4b537113', 'request_id': 'req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:17.660134 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b252456d95d94021804c380e4b537113', 'request_id': 'req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:17.760160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b252456d95d94021804c380e4b537113', 'request_id': 'req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:17.771598 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2 admin admin] The token cache is being invalidate because project b252456d95d94021804c380e4b537113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:37:17.771898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:17.772537 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:17.774040 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1154/2278] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:37:17 2022] DELETE /identity/v3/projects/b252456d95d94021804c380e4b537113 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:17.774345 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4e4d5976-d39a-4334-9cbc-6f4e9165a2d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44a68887d49449e4a4b3911d14a03ed6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:17.863100 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4e4d5976-d39a-4334-9cbc-6f4e9165a2d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44a68887d49449e4a4b3911d14a03ed6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:17.863749 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4e4d5976-d39a-4334-9cbc-6f4e9165a2d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44a68887d49449e4a4b3911d14a03ed6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:17.882053 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4e4d5976-d39a-4334-9cbc-6f4e9165a2d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44a68887d49449e4a4b3911d14a03ed6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:17.892552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4e4d5976-d39a-4334-9cbc-6f4e9165a2d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44a68887d49449e4a4b3911d14a03ed6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:17.899158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1125/2279] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:37:17 2022] DELETE /identity/v3/users/44a68887d49449e4a4b3911d14a03ed6 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:37:18.388999 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:18.391122 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:18.444031 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:18.461514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354c42b-ed1d-48c4-8b10-29ff7089b8e2 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:18.528256 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b009251-48f8-468d-81f8-fc370601c7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:18.528256 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b009251-48f8-468d-81f8-fc370601c7b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:18.528256 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b009251-48f8-468d-81f8-fc370601c7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:18.528256 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b009251-48f8-468d-81f8-fc370601c7b7 admin admin] PATH_INFO: `/v3/projects/6bf138ab196043f9bacc2cfb15e8dc44` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:18.528674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b009251-48f8-468d-81f8-fc370601c7b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bf138ab196043f9bacc2cfb15e8dc44)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:18.530085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b009251-48f8-468d-81f8-fc370601c7b7 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:18.657671 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-6b009251-48f8-468d-81f8-fc370601c7b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bf138ab196043f9bacc2cfb15e8dc44', 'request_id': 'req-6b009251-48f8-468d-81f8-fc370601c7b7'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:18.663814 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-6b009251-48f8-468d-81f8-fc370601c7b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bf138ab196043f9bacc2cfb15e8dc44', 'request_id': 'req-6b009251-48f8-468d-81f8-fc370601c7b7'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:18.794194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-6b009251-48f8-468d-81f8-fc370601c7b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bf138ab196043f9bacc2cfb15e8dc44', 'request_id': 'req-6b009251-48f8-468d-81f8-fc370601c7b7'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:18.805702 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-6b009251-48f8-468d-81f8-fc370601c7b7 admin admin] The token cache is being invalidate because project 6bf138ab196043f9bacc2cfb15e8dc44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:37:18.805989 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-6b009251-48f8-468d-81f8-fc370601c7b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:18.806609 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-6b009251-48f8-468d-81f8-fc370601c7b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:18.808118 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1155/2280] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:37:18 2022] DELETE /identity/v3/projects/6bf138ab196043f9bacc2cfb15e8dc44 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:18.813076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af7e90-589e-40c9-90c9-81c5d82a2937 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:18.813552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af7e90-589e-40c9-90c9-81c5d82a2937 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:18.813657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af7e90-589e-40c9-90c9-81c5d82a2937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:18.833737 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-f8af7e90-589e-40c9-90c9-81c5d82a2937 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:18.836903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f8af7e90-589e-40c9-90c9-81c5d82a2937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:18.904411 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1126/2281] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:37:18.913779 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3367c7-0fb6-4093-b309-3e1e3d25cea4 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:18.914066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3367c7-0fb6-4093-b309-3e1e3d25cea4 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:18.914341 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3367c7-0fb6-4093-b309-3e1e3d25cea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:18.935638 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-2b3367c7-0fb6-4093-b309-3e1e3d25cea4 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:18.936851 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-2b3367c7-0fb6-4093-b309-3e1e3d25cea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:18.943047 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:18.944712 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1156/2282] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:37:18.948778 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af7e90-589e-40c9-90c9-81c5d82a2937 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:18.958494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af7e90-589e-40c9-90c9-81c5d82a2937 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:18.963066 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca86bf5-b89e-4a93-b3d1-7dae5f075b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:18.963663 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca86bf5-b89e-4a93-b3d1-7dae5f075b96 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:18.963994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca86bf5-b89e-4a93-b3d1-7dae5f075b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:18.964275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca86bf5-b89e-4a93-b3d1-7dae5f075b96 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:18.964921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca86bf5-b89e-4a93-b3d1-7dae5f075b96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:37:18.965231 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca86bf5-b89e-4a93-b3d1-7dae5f075b96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:18.967162 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca86bf5-b89e-4a93-b3d1-7dae5f075b96 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:18.978991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1127/2283] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:37:18.983085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3367c7-0fb6-4093-b309-3e1e3d25cea4 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:18.992734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3367c7-0fb6-4093-b309-3e1e3d25cea4 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:18.997303 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c14097-cc95-4d1c-9de0-a88a705229a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:18.997891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c14097-cc95-4d1c-9de0-a88a705229a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:18.998169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c14097-cc95-4d1c-9de0-a88a705229a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:18.998439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c14097-cc95-4d1c-9de0-a88a705229a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:18.999100 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c14097-cc95-4d1c-9de0-a88a705229a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:19.001044 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c14097-cc95-4d1c-9de0-a88a705229a9 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:19.146722 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1157/2284] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:37:18 2022] POST /identity/v3/projects => generated 389 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:37:19.148330 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca86bf5-b89e-4a93-b3d1-7dae5f075b96 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:19.158129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca86bf5-b89e-4a93-b3d1-7dae5f075b96 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:19.214121 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247419e8-acbc-446d-b7f1-abe0aeb7399b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:19.214731 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247419e8-acbc-446d-b7f1-abe0aeb7399b admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:19.215099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247419e8-acbc-446d-b7f1-abe0aeb7399b admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:19.215359 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247419e8-acbc-446d-b7f1-abe0aeb7399b admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:19.216002 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247419e8-acbc-446d-b7f1-abe0aeb7399b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:19.218196 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247419e8-acbc-446d-b7f1-abe0aeb7399b admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:19.371754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1128/2285] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:37:19 2022] POST /identity/v3/users => generated 295 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:37:19.376396 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c14097-cc95-4d1c-9de0-a88a705229a9 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:19.386926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c14097-cc95-4d1c-9de0-a88a705229a9 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:19.391715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac8cc9-e3c1-4e11-80d3-e6e3bafbc312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:19.392320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac8cc9-e3c1-4e11-80d3-e6e3bafbc312 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:19.392598 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac8cc9-e3c1-4e11-80d3-e6e3bafbc312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:19.392870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac8cc9-e3c1-4e11-80d3-e6e3bafbc312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:19.393552 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac8cc9-e3c1-4e11-80d3-e6e3bafbc312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:19.395216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac8cc9-e3c1-4e11-80d3-e6e3bafbc312 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:19.408095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1158/2286] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:37:19.413392 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247419e8-acbc-446d-b7f1-abe0aeb7399b admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:19.424090 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247419e8-acbc-446d-b7f1-abe0aeb7399b admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:19.428844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724d197-3cd7-4637-802d-d0f125399bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:19.429444 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724d197-3cd7-4637-802d-d0f125399bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:19.429729 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724d197-3cd7-4637-802d-d0f125399bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:19.430000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724d197-3cd7-4637-802d-d0f125399bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:19.430639 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724d197-3cd7-4637-802d-d0f125399bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:19.432370 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724d197-3cd7-4637-802d-d0f125399bc3 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:19.447322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1129/2287] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:37:19.451948 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac8cc9-e3c1-4e11-80d3-e6e3bafbc312 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:19.464230 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac8cc9-e3c1-4e11-80d3-e6e3bafbc312 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:19.474237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189df52c-43f1-4efe-97f9-dadff17c5cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:19.474958 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189df52c-43f1-4efe-97f9-dadff17c5cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:19.475239 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189df52c-43f1-4efe-97f9-dadff17c5cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:19.475514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189df52c-43f1-4efe-97f9-dadff17c5cd5 admin admin] PATH_INFO: `/v3/projects/56e7fbc6857d4b01abd233a04536065a/users/5d42cbcaf6ab411790e2737c10a474d6/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:19.491192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189df52c-43f1-4efe-97f9-dadff17c5cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56e7fbc6857d4b01abd233a04536065a, user_id=5d42cbcaf6ab411790e2737c10a474d6, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:19.493913 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189df52c-43f1-4efe-97f9-dadff17c5cd5 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:19.612049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1159/2288] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:37:19 2022] PUT /identity/v3/projects/56e7fbc6857d4b01abd233a04536065a/users/5d42cbcaf6ab411790e2737c10a474d6/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:19.626714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885faa2b-6470-465d-8e18-c820eda8d136 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:19.626982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885faa2b-6470-465d-8e18-c820eda8d136 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:19.627323 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885faa2b-6470-465d-8e18-c820eda8d136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:19.656174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-885faa2b-6470-465d-8e18-c820eda8d136 None None] MFA Rules not processed for user `5d42cbcaf6ab411790e2737c10a474d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:19.657468 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-885faa2b-6470-465d-8e18-c820eda8d136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:19.720703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1130/2289] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:37:19.731223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189df52c-43f1-4efe-97f9-dadff17c5cd5 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:19.732901 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-189df52c-43f1-4efe-97f9-dadff17c5cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:19.754724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:19.790260 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189df52c-43f1-4efe-97f9-dadff17c5cd5 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:19.793893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea034bc3-1794-4dd2-9346-2c7376f3c3c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:19.794493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea034bc3-1794-4dd2-9346-2c7376f3c3c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:19.808605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea034bc3-1794-4dd2-9346-2c7376f3c3c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:19.808605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea034bc3-1794-4dd2-9346-2c7376f3c3c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:19.808605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea034bc3-1794-4dd2-9346-2c7376f3c3c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:19.808605 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea034bc3-1794-4dd2-9346-2c7376f3c3c9 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:19.814166 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1160/2290] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:37:19 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:37:19.976332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:20.376503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:21.217540 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:21.245889 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:21.400225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:21.423399 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:21.520043 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:21.617206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:21.657367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:21.771521 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:21.811685 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:21.971645 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:22.005747 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:22.160399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:22.189438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=78,keystone:DELETE=18,keystone:INSERT=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:22.360114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:22.394079 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:22.617746 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:22.640008 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:22.828620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:22.862994 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:22.936604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:22.965938 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:23.080233 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:23.257114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:23.296902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:23.473119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:23.508518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:23.702763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:23.737541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:23.908646 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:26.625569 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885faa2b-6470-465d-8e18-c820eda8d136 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:26.627030 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-885faa2b-6470-465d-8e18-c820eda8d136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:26.644765 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:26.695779 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885faa2b-6470-465d-8e18-c820eda8d136 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:26.700699 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f974d-6bec-44b2-ae8e-0b842ec3f15a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:26.701302 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f974d-6bec-44b2-ae8e-0b842ec3f15a service placement] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:26.702139 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f974d-6bec-44b2-ae8e-0b842ec3f15a service placement] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:26.702421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f974d-6bec-44b2-ae8e-0b842ec3f15a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:26.704200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-672f974d-6bec-44b2-ae8e-0b842ec3f15a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:26.715789 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f974d-6bec-44b2-ae8e-0b842ec3f15a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:26.717627 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f974d-6bec-44b2-ae8e-0b842ec3f15a service placement] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:26.727351 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1131/2291] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:37:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:37:33.948304 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:36.723213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=25 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:45.933649 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea034bc3-1794-4dd2-9346-2c7376f3c3c9 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:45.935204 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ea034bc3-1794-4dd2-9346-2c7376f3c3c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:46.001784 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea034bc3-1794-4dd2-9346-2c7376f3c3c9 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:46.006558 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560b052-8d3e-4ccc-8104-87901eea16c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:46.007223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560b052-8d3e-4ccc-8104-87901eea16c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:46.007505 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560b052-8d3e-4ccc-8104-87901eea16c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:46.007782 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560b052-8d3e-4ccc-8104-87901eea16c6 admin admin] PATH_INFO: `/v3/users/2222fcfe1f5c4dc5a3c3f08c63d301b1` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:46.008909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560b052-8d3e-4ccc-8104-87901eea16c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2222fcfe1f5c4dc5a3c3f08c63d301b1)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:46.010847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560b052-8d3e-4ccc-8104-87901eea16c6 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:46.207250 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-7560b052-8d3e-4ccc-8104-87901eea16c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2222fcfe1f5c4dc5a3c3f08c63d301b1'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:46.279620 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-7560b052-8d3e-4ccc-8104-87901eea16c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2222fcfe1f5c4dc5a3c3f08c63d301b1'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:46.292833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-7560b052-8d3e-4ccc-8104-87901eea16c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2222fcfe1f5c4dc5a3c3f08c63d301b1'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:46.293472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-7560b052-8d3e-4ccc-8104-87901eea16c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2222fcfe1f5c4dc5a3c3f08c63d301b1'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:46.307130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-7560b052-8d3e-4ccc-8104-87901eea16c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2222fcfe1f5c4dc5a3c3f08c63d301b1'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:46.307130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1161/2292] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:37:45 2022] DELETE /identity/v3/users/2222fcfe1f5c4dc5a3c3f08c63d301b1 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:46.311003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef670e-4f03-4a96-a1d9-0d87ea95e2f3 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:46.311453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef670e-4f03-4a96-a1d9-0d87ea95e2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:46.311568 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef670e-4f03-4a96-a1d9-0d87ea95e2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:46.331564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-0eef670e-4f03-4a96-a1d9-0d87ea95e2f3 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:46.332753 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-0eef670e-4f03-4a96-a1d9-0d87ea95e2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:46.391190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1132/2293] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:37:46.400934 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560b052-8d3e-4ccc-8104-87901eea16c6 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:46.402349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7560b052-8d3e-4ccc-8104-87901eea16c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:46.465629 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560b052-8d3e-4ccc-8104-87901eea16c6 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:46.470924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882605c-057b-4123-8b1d-1fcd0b7d67bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:46.471586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882605c-057b-4123-8b1d-1fcd0b7d67bc service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:46.471962 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882605c-057b-4123-8b1d-1fcd0b7d67bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:46.472237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882605c-057b-4123-8b1d-1fcd0b7d67bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:46.481750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882605c-057b-4123-8b1d-1fcd0b7d67bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:46.483220 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882605c-057b-4123-8b1d-1fcd0b7d67bc service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:46.493322 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1162/2294] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:37:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:37:46.932600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef670e-4f03-4a96-a1d9-0d87ea95e2f3 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:46.934299 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-0eef670e-4f03-4a96-a1d9-0d87ea95e2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:46.949368 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef670e-4f03-4a96-a1d9-0d87ea95e2f3 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:46.954668 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7899a-a026-403f-baa3-62496c59f32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:46.955340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7899a-a026-403f-baa3-62496c59f32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:46.955622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7899a-a026-403f-baa3-62496c59f32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:46.956645 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7899a-a026-403f-baa3-62496c59f32e admin admin] PATH_INFO: `/v3/projects/a837cc5e269446f597dc6b8c44dacf39` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:46.958450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7899a-a026-403f-baa3-62496c59f32e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a837cc5e269446f597dc6b8c44dacf39)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:46.961063 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7899a-a026-403f-baa3-62496c59f32e admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:47.084809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f5a7899a-a026-403f-baa3-62496c59f32e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a837cc5e269446f597dc6b8c44dacf39', 'request_id': 'req-f5a7899a-a026-403f-baa3-62496c59f32e'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:47.091091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f5a7899a-a026-403f-baa3-62496c59f32e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a837cc5e269446f597dc6b8c44dacf39', 'request_id': 'req-f5a7899a-a026-403f-baa3-62496c59f32e'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:47.193398 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f5a7899a-a026-403f-baa3-62496c59f32e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a837cc5e269446f597dc6b8c44dacf39', 'request_id': 'req-f5a7899a-a026-403f-baa3-62496c59f32e'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:47.204905 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f5a7899a-a026-403f-baa3-62496c59f32e admin admin] The token cache is being invalidate because project a837cc5e269446f597dc6b8c44dacf39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:37:47.205193 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f5a7899a-a026-403f-baa3-62496c59f32e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:47.205819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f5a7899a-a026-403f-baa3-62496c59f32e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:47.207332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1133/2295] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:37:46 2022] DELETE /identity/v3/projects/a837cc5e269446f597dc6b8c44dacf39 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:56.489068 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:57.201823 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:57.405313 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f394411-8611-44c3-9ca4-ea84d596d402 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:57.405582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f394411-8611-44c3-9ca4-ea84d596d402 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:57.405863 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f394411-8611-44c3-9ca4-ea84d596d402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:57.425821 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-5f394411-8611-44c3-9ca4-ea84d596d402 None None] MFA Rules not processed for user `86300433c724483994f59709a3f015db`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:57.427104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5f394411-8611-44c3-9ca4-ea84d596d402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:57.485077 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1163/2296] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 5 14:37:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:37:57.527725 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7899a-a026-403f-baa3-62496c59f32e admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:57.529567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f5a7899a-a026-403f-baa3-62496c59f32e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:57.600941 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7899a-a026-403f-baa3-62496c59f32e admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:57.606186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0f196-50ec-4978-9e75-9945744a29bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a85809d14cb449d848f22a3f39bf641', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:57.606828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0f196-50ec-4978-9e75-9945744a29bc service swift] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:57.607115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0f196-50ec-4978-9e75-9945744a29bc service swift] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:57.607391 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0f196-50ec-4978-9e75-9945744a29bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:57.616046 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0f196-50ec-4978-9e75-9945744a29bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:57.617621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0f196-50ec-4978-9e75-9945744a29bc service swift] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:57.626742 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1134/2297] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 5 14:37:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 14:37:57.649607 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f394411-8611-44c3-9ca4-ea84d596d402 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:57.651816 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5f394411-8611-44c3-9ca4-ea84d596d402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:57.746284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f394411-8611-44c3-9ca4-ea84d596d402 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:57.752630 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789a152-d188-47d7-a5b9-2eb448c3e664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:57.753631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789a152-d188-47d7-a5b9-2eb448c3e664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:57.753973 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789a152-d188-47d7-a5b9-2eb448c3e664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:57.754423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789a152-d188-47d7-a5b9-2eb448c3e664 admin admin] PATH_INFO: `/v3/users/7bebac1e3eac4f21a531843bdcd0e8d0` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:57.755984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789a152-d188-47d7-a5b9-2eb448c3e664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bebac1e3eac4f21a531843bdcd0e8d0)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:57.758707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789a152-d188-47d7-a5b9-2eb448c3e664 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:57.959219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8789a152-d188-47d7-a5b9-2eb448c3e664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bebac1e3eac4f21a531843bdcd0e8d0'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:58.018166 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8789a152-d188-47d7-a5b9-2eb448c3e664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bebac1e3eac4f21a531843bdcd0e8d0'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:58.034811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8789a152-d188-47d7-a5b9-2eb448c3e664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bebac1e3eac4f21a531843bdcd0e8d0'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:58.035671 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8789a152-d188-47d7-a5b9-2eb448c3e664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bebac1e3eac4f21a531843bdcd0e8d0'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:58.043930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8789a152-d188-47d7-a5b9-2eb448c3e664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bebac1e3eac4f21a531843bdcd0e8d0'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:58.050263 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1164/2298] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:37:57 2022] DELETE /identity/v3/users/7bebac1e3eac4f21a531843bdcd0e8d0 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:58.054750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171d0a1-0b68-4049-a867-b5d0ef2e1d2b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:58.055122 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171d0a1-0b68-4049-a867-b5d0ef2e1d2b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:58.055349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171d0a1-0b68-4049-a867-b5d0ef2e1d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:58.075320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-c171d0a1-0b68-4049-a867-b5d0ef2e1d2b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:58.078992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c171d0a1-0b68-4049-a867-b5d0ef2e1d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:58.149406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1135/2299] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:37:58.160748 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789a152-d188-47d7-a5b9-2eb448c3e664 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:58.162385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8789a152-d188-47d7-a5b9-2eb448c3e664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:58.200655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:58.244493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789a152-d188-47d7-a5b9-2eb448c3e664 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:58.253011 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026433f-54ab-4b0c-ab2f-054b42112ffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:58.253011 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026433f-54ab-4b0c-ab2f-054b42112ffb service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:58.253011 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026433f-54ab-4b0c-ab2f-054b42112ffb service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:58.253011 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026433f-54ab-4b0c-ab2f-054b42112ffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:58.264655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026433f-54ab-4b0c-ab2f-054b42112ffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:58.266473 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026433f-54ab-4b0c-ab2f-054b42112ffb service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:58.277478 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1165/2300] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:37:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:37:58.767074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171d0a1-0b68-4049-a867-b5d0ef2e1d2b None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:58.769118 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c171d0a1-0b68-4049-a867-b5d0ef2e1d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:58.783899 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171d0a1-0b68-4049-a867-b5d0ef2e1d2b None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:58.790342 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbfb53-2e2d-4602-be80-a3bf43ddf655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:58.791330 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbfb53-2e2d-4602-be80-a3bf43ddf655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:58.792117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbfb53-2e2d-4602-be80-a3bf43ddf655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:58.792637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbfb53-2e2d-4602-be80-a3bf43ddf655 admin admin] PATH_INFO: `/v3/projects/1382cd3884c343df83d0b2dffd483e77` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:58.794069 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbfb53-2e2d-4602-be80-a3bf43ddf655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1382cd3884c343df83d0b2dffd483e77)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:58.795883 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbfb53-2e2d-4602-be80-a3bf43ddf655 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:58.928025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e1bbfb53-2e2d-4602-be80-a3bf43ddf655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1382cd3884c343df83d0b2dffd483e77', 'request_id': 'req-e1bbfb53-2e2d-4602-be80-a3bf43ddf655'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:58.941203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e1bbfb53-2e2d-4602-be80-a3bf43ddf655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1382cd3884c343df83d0b2dffd483e77', 'request_id': 'req-e1bbfb53-2e2d-4602-be80-a3bf43ddf655'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:59.045217 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e1bbfb53-2e2d-4602-be80-a3bf43ddf655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1382cd3884c343df83d0b2dffd483e77', 'request_id': 'req-e1bbfb53-2e2d-4602-be80-a3bf43ddf655'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:59.058261 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e1bbfb53-2e2d-4602-be80-a3bf43ddf655 admin admin] The token cache is being invalidate because project 1382cd3884c343df83d0b2dffd483e77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:37:59.058557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e1bbfb53-2e2d-4602-be80-a3bf43ddf655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:59.059320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e1bbfb53-2e2d-4602-be80-a3bf43ddf655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:37:59.065091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1136/2301] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:37:58 2022] DELETE /identity/v3/projects/1382cd3884c343df83d0b2dffd483e77 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:37:59.067434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fef49-bbb6-4731-a57a-b1ad73cd78e8 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:59.068336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fef49-bbb6-4731-a57a-b1ad73cd78e8 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:59.068787 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fef49-bbb6-4731-a57a-b1ad73cd78e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:59.107158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-5f7fef49-bbb6-4731-a57a-b1ad73cd78e8 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:59.108529 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5f7fef49-bbb6-4731-a57a-b1ad73cd78e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:59.194876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1166/2302] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:37:59.201211 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f25b0-2c2f-4180-8a03-211093795b68 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:59.201697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f25b0-2c2f-4180-8a03-211093795b68 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:59.202151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f25b0-2c2f-4180-8a03-211093795b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:59.226175 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-532f25b0-2c2f-4180-8a03-211093795b68 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:37:59.227932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-532f25b0-2c2f-4180-8a03-211093795b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:37:59.236696 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1137/2303] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:37:59.241085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7fef49-bbb6-4731-a57a-b1ad73cd78e8 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:59.252378 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7fef49-bbb6-4731-a57a-b1ad73cd78e8 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:59.257168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6af74c-5d9c-4d2b-ae7b-5e6dad8c26bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:59.259853 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6af74c-5d9c-4d2b-ae7b-5e6dad8c26bf admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:59.259853 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6af74c-5d9c-4d2b-ae7b-5e6dad8c26bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:59.259853 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6af74c-5d9c-4d2b-ae7b-5e6dad8c26bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:59.259853 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6af74c-5d9c-4d2b-ae7b-5e6dad8c26bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:37:59.259853 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6af74c-5d9c-4d2b-ae7b-5e6dad8c26bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:59.260893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6af74c-5d9c-4d2b-ae7b-5e6dad8c26bf admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:59.274349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1167/2304] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:37:59.278909 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f25b0-2c2f-4180-8a03-211093795b68 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:59.289017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f25b0-2c2f-4180-8a03-211093795b68 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:59.294415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57610d-006c-4971-9b1f-e5e4d9d04d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:59.295099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57610d-006c-4971-9b1f-e5e4d9d04d70 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:59.295387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57610d-006c-4971-9b1f-e5e4d9d04d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:59.295663 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57610d-006c-4971-9b1f-e5e4d9d04d70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:59.296338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57610d-006c-4971-9b1f-e5e4d9d04d70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:59.298425 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57610d-006c-4971-9b1f-e5e4d9d04d70 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:59.303033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:37:59.602084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1138/2305] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:37:59 2022] POST /identity/v3/projects => generated 369 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:37:59.605193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6af74c-5d9c-4d2b-ae7b-5e6dad8c26bf admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:59.624997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6af74c-5d9c-4d2b-ae7b-5e6dad8c26bf admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:59.731565 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5afaf1-845e-4143-a626-e02a6fc61146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:59.732678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5afaf1-845e-4143-a626-e02a6fc61146 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:59.732912 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5afaf1-845e-4143-a626-e02a6fc61146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:59.733302 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5afaf1-845e-4143-a626-e02a6fc61146 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:59.734337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5afaf1-845e-4143-a626-e02a6fc61146 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:59.736382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5afaf1-845e-4143-a626-e02a6fc61146 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:59.875753 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1168/2306] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:37:59 2022] POST /identity/v3/users => generated 285 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:37:59.880595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57610d-006c-4971-9b1f-e5e4d9d04d70 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:59.895947 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57610d-006c-4971-9b1f-e5e4d9d04d70 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:59.901864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20f171-9a48-4ea8-9896-7e0e12fb1572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:59.902556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20f171-9a48-4ea8-9896-7e0e12fb1572 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:59.902898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20f171-9a48-4ea8-9896-7e0e12fb1572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:59.903229 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20f171-9a48-4ea8-9896-7e0e12fb1572 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:59.904320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb20f171-9a48-4ea8-9896-7e0e12fb1572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:59.907666 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb20f171-9a48-4ea8-9896-7e0e12fb1572 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:59.919282 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1139/2307] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:37:59.928016 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5afaf1-845e-4143-a626-e02a6fc61146 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:59.939061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5afaf1-845e-4143-a626-e02a6fc61146 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:59.945907 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff088fe9-8d37-40f0-9059-46e0fa20033c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:59.946677 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff088fe9-8d37-40f0-9059-46e0fa20033c admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:59.946991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff088fe9-8d37-40f0-9059-46e0fa20033c admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:59.947272 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff088fe9-8d37-40f0-9059-46e0fa20033c admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:37:59.948046 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff088fe9-8d37-40f0-9059-46e0fa20033c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:37:59.950114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff088fe9-8d37-40f0-9059-46e0fa20033c admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:37:59.965272 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1169/2308] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:37:59.969897 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20f171-9a48-4ea8-9896-7e0e12fb1572 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:37:59.982500 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20f171-9a48-4ea8-9896-7e0e12fb1572 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:37:59.987809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dd80b-fa7f-4a7e-9f07-7f90dd33a9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:37:59.988493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dd80b-fa7f-4a7e-9f07-7f90dd33a9cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:37:59.988862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dd80b-fa7f-4a7e-9f07-7f90dd33a9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:37:59.989163 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dd80b-fa7f-4a7e-9f07-7f90dd33a9cd admin admin] PATH_INFO: `/v3/projects/ef79d84f54ff42599908ca6c184805ab/users/8822d0103d8e40c0961b8cf6c4b4e531/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:00.005854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028dd80b-fa7f-4a7e-9f07-7f90dd33a9cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef79d84f54ff42599908ca6c184805ab, user_id=8822d0103d8e40c0961b8cf6c4b4e531, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:00.008644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028dd80b-fa7f-4a7e-9f07-7f90dd33a9cd admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:00.086901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1140/2309] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:37:59 2022] PUT /identity/v3/projects/ef79d84f54ff42599908ca6c184805ab/users/8822d0103d8e40c0961b8cf6c4b4e531/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:38:00.097789 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadff89a-c15f-4fa6-8ca4-f0034232cbc1 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:00.099000 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadff89a-c15f-4fa6-8ca4-f0034232cbc1 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:00.099384 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadff89a-c15f-4fa6-8ca4-f0034232cbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:00.147999 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-cadff89a-c15f-4fa6-8ca4-f0034232cbc1 None None] MFA Rules not processed for user `8822d0103d8e40c0961b8cf6c4b4e531`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:00.149417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-cadff89a-c15f-4fa6-8ca4-f0034232cbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:00.174818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:38:00.260083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1170/2310] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:38:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:38:00.276010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dd80b-fa7f-4a7e-9f07-7f90dd33a9cd admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:00.282105 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-028dd80b-fa7f-4a7e-9f07-7f90dd33a9cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:00.336339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dd80b-fa7f-4a7e-9f07-7f90dd33a9cd admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:00.340064 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0463d-c75b-4468-8233-37d88932ea15 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '735986478c354ed196c601f12f3ac377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:00.340675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0463d-c75b-4468-8233-37d88932ea15 service glance] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:00.340962 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0463d-c75b-4468-8233-37d88932ea15 service glance] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:00.341240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0463d-c75b-4468-8233-37d88932ea15 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:00.350115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0463d-c75b-4468-8233-37d88932ea15 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:00.351612 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0463d-c75b-4468-8233-37d88932ea15 service glance] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:00.361182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1141/2311] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:38:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:38:00.875903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadff89a-c15f-4fa6-8ca4-f0034232cbc1 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:00.877463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-cadff89a-c15f-4fa6-8ca4-f0034232cbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:00.965357 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadff89a-c15f-4fa6-8ca4-f0034232cbc1 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:00.971661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fc5bc-b029-4722-9551-1be4238b8aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:00.972485 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fc5bc-b029-4722-9551-1be4238b8aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:00.972907 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fc5bc-b029-4722-9551-1be4238b8aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:00.973340 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fc5bc-b029-4722-9551-1be4238b8aa1 admin admin] PATH_INFO: `/v3/users/5d42cbcaf6ab411790e2737c10a474d6` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:00.974537 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25fc5bc-b029-4722-9551-1be4238b8aa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d42cbcaf6ab411790e2737c10a474d6)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:00.976818 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25fc5bc-b029-4722-9551-1be4238b8aa1 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:01.219569 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e25fc5bc-b029-4722-9551-1be4238b8aa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d42cbcaf6ab411790e2737c10a474d6'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:01.304321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e25fc5bc-b029-4722-9551-1be4238b8aa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d42cbcaf6ab411790e2737c10a474d6'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:01.325019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e25fc5bc-b029-4722-9551-1be4238b8aa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d42cbcaf6ab411790e2737c10a474d6'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:01.326606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e25fc5bc-b029-4722-9551-1be4238b8aa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d42cbcaf6ab411790e2737c10a474d6'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:01.340995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e25fc5bc-b029-4722-9551-1be4238b8aa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d42cbcaf6ab411790e2737c10a474d6'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:01.355615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1171/2312] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:38:00 2022] DELETE /identity/v3/users/5d42cbcaf6ab411790e2737c10a474d6 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:38:01.358120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22df34-90b7-43fb-932b-a51aeb3944fe None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:01.359088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22df34-90b7-43fb-932b-a51aeb3944fe None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:01.359314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22df34-90b7-43fb-932b-a51aeb3944fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:01.389466 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-0f22df34-90b7-43fb-932b-a51aeb3944fe None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:01.391478 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-0f22df34-90b7-43fb-932b-a51aeb3944fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:01.462360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1142/2313] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:38:01.478619 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fc5bc-b029-4722-9551-1be4238b8aa1 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:01.480355 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e25fc5bc-b029-4722-9551-1be4238b8aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:01.535774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:38:01.549969 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fc5bc-b029-4722-9551-1be4238b8aa1 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:01.553877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a80d3-ca32-4901-a478-787db1f5905f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:01.554497 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a80d3-ca32-4901-a478-787db1f5905f service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:01.554854 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a80d3-ca32-4901-a478-787db1f5905f service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:01.555119 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a80d3-ca32-4901-a478-787db1f5905f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:01.565988 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a80d3-ca32-4901-a478-787db1f5905f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:01.567819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a80d3-ca32-4901-a478-787db1f5905f service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:01.580489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1172/2314] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:38:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:38:02.060218 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22df34-90b7-43fb-932b-a51aeb3944fe None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:02.060218 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-0f22df34-90b7-43fb-932b-a51aeb3944fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:02.082208 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22df34-90b7-43fb-932b-a51aeb3944fe None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:02.088102 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1b523-fc48-4ecc-814f-318cd06814ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:02.089080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b523-fc48-4ecc-814f-318cd06814ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:02.089584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b523-fc48-4ecc-814f-318cd06814ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:02.090069 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b523-fc48-4ecc-814f-318cd06814ba admin admin] PATH_INFO: `/v3/projects/56e7fbc6857d4b01abd233a04536065a` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:02.096078 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1b523-fc48-4ecc-814f-318cd06814ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=56e7fbc6857d4b01abd233a04536065a)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:02.098369 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1b523-fc48-4ecc-814f-318cd06814ba admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:02.229582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:38:02.231024 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2fa1b523-fc48-4ecc-814f-318cd06814ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56e7fbc6857d4b01abd233a04536065a', 'request_id': 'req-2fa1b523-fc48-4ecc-814f-318cd06814ba'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:02.237080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2fa1b523-fc48-4ecc-814f-318cd06814ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56e7fbc6857d4b01abd233a04536065a', 'request_id': 'req-2fa1b523-fc48-4ecc-814f-318cd06814ba'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:02.314124 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2fa1b523-fc48-4ecc-814f-318cd06814ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56e7fbc6857d4b01abd233a04536065a', 'request_id': 'req-2fa1b523-fc48-4ecc-814f-318cd06814ba'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:02.326488 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2fa1b523-fc48-4ecc-814f-318cd06814ba admin admin] The token cache is being invalidate because project 56e7fbc6857d4b01abd233a04536065a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:38:02.327037 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2fa1b523-fc48-4ecc-814f-318cd06814ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:02.328293 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2fa1b523-fc48-4ecc-814f-318cd06814ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:02.329778 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1143/2315] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:38:02 2022] DELETE /identity/v3/projects/56e7fbc6857d4b01abd233a04536065a => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:38:02.334943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdcd77-b79b-44f7-9e53-d8653f774c6c None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:02.335438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdcd77-b79b-44f7-9e53-d8653f774c6c None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:02.335554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdcd77-b79b-44f7-9e53-d8653f774c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:02.356049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-94cdcd77-b79b-44f7-9e53-d8653f774c6c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:02.363438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-94cdcd77-b79b-44f7-9e53-d8653f774c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:02.443355 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef350435-ad62-453c-a3a2-12948f8c553b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:02.443830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef350435-ad62-453c-a3a2-12948f8c553b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:02.444029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef350435-ad62-453c-a3a2-12948f8c553b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:02.450312 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1173/2316] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:38:02.471179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-ef350435-ad62-453c-a3a2-12948f8c553b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:02.473449 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ef350435-ad62-453c-a3a2-12948f8c553b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:02.487309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1144/2317] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:38:02.494315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cdcd77-b79b-44f7-9e53-d8653f774c6c None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:02.508052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cdcd77-b79b-44f7-9e53-d8653f774c6c None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:02.514009 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c34b3-5b93-47d6-a2cc-c11174aed457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:02.514625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c34b3-5b93-47d6-a2cc-c11174aed457 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:02.515779 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c34b3-5b93-47d6-a2cc-c11174aed457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:02.516069 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c34b3-5b93-47d6-a2cc-c11174aed457 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:02.516755 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c34b3-5b93-47d6-a2cc-c11174aed457 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:38:02.517088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c34b3-5b93-47d6-a2cc-c11174aed457 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:02.518727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c34b3-5b93-47d6-a2cc-c11174aed457 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:02.532068 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1174/2318] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:38:02.536373 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef350435-ad62-453c-a3a2-12948f8c553b None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:02.551149 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef350435-ad62-453c-a3a2-12948f8c553b None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:02.560161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e60029-8f3c-4dec-bc8f-72555d9ee08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:02.560777 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e60029-8f3c-4dec-bc8f-72555d9ee08c admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:02.561265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e60029-8f3c-4dec-bc8f-72555d9ee08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:02.561541 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e60029-8f3c-4dec-bc8f-72555d9ee08c admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:02.562728 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e60029-8f3c-4dec-bc8f-72555d9ee08c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:02.564898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e60029-8f3c-4dec-bc8f-72555d9ee08c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:02.700523 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1145/2319] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:38:02 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:38:02.704893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c34b3-5b93-47d6-a2cc-c11174aed457 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:02.715417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c34b3-5b93-47d6-a2cc-c11174aed457 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:02.789066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2dfac4-5b8e-4b32-a5ec-47486fb69940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:02.789719 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2dfac4-5b8e-4b32-a5ec-47486fb69940 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:02.790002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2dfac4-5b8e-4b32-a5ec-47486fb69940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:02.790283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2dfac4-5b8e-4b32-a5ec-47486fb69940 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:02.790954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2dfac4-5b8e-4b32-a5ec-47486fb69940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:02.792944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2dfac4-5b8e-4b32-a5ec-47486fb69940 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:02.899270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1175/2320] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:38:02 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:38:02.973105 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e60029-8f3c-4dec-bc8f-72555d9ee08c admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:02.984278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e60029-8f3c-4dec-bc8f-72555d9ee08c admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:02.989699 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ce81b-2db1-489e-a411-0df2b23a72d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:02.990329 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ce81b-2db1-489e-a411-0df2b23a72d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:02.990613 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ce81b-2db1-489e-a411-0df2b23a72d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:02.991009 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ce81b-2db1-489e-a411-0df2b23a72d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:03.000472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ce81b-2db1-489e-a411-0df2b23a72d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:03.000472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ce81b-2db1-489e-a411-0df2b23a72d3 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:03.006819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1146/2321] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:38:03.009967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2dfac4-5b8e-4b32-a5ec-47486fb69940 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:03.022208 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2dfac4-5b8e-4b32-a5ec-47486fb69940 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:03.027390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464ceb1-f7e1-4786-abe7-48158d87e8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:03.029163 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464ceb1-f7e1-4786-abe7-48158d87e8a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:03.029163 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464ceb1-f7e1-4786-abe7-48158d87e8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:03.029163 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464ceb1-f7e1-4786-abe7-48158d87e8a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:03.029527 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464ceb1-f7e1-4786-abe7-48158d87e8a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:03.031099 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464ceb1-f7e1-4786-abe7-48158d87e8a2 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:03.044273 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1176/2322] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:38:03.048644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ce81b-2db1-489e-a411-0df2b23a72d3 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:03.061329 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ce81b-2db1-489e-a411-0df2b23a72d3 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:03.067014 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28084557-8dba-43dc-bbfd-ac283f034ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:03.067969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28084557-8dba-43dc-bbfd-ac283f034ac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:03.068210 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28084557-8dba-43dc-bbfd-ac283f034ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:03.068490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28084557-8dba-43dc-bbfd-ac283f034ac0 admin admin] PATH_INFO: `/v3/projects/e17efecb1ba44e69ac6b076e56ec9603/users/1a2d7c64ad9d44f997333ace9d4e60a1/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:03.089629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28084557-8dba-43dc-bbfd-ac283f034ac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e17efecb1ba44e69ac6b076e56ec9603, user_id=1a2d7c64ad9d44f997333ace9d4e60a1, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:03.093257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28084557-8dba-43dc-bbfd-ac283f034ac0 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:03.184390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1147/2323] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:38:03 2022] PUT /identity/v3/projects/e17efecb1ba44e69ac6b076e56ec9603/users/1a2d7c64ad9d44f997333ace9d4e60a1/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:38:03.190406 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5805c6-1db6-44f7-89b8-9013f9968f9f None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:03.190825 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5805c6-1db6-44f7-89b8-9013f9968f9f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:03.191201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5805c6-1db6-44f7-89b8-9013f9968f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:03.212324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-bc5805c6-1db6-44f7-89b8-9013f9968f9f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:03.213878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-bc5805c6-1db6-44f7-89b8-9013f9968f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:03.301166 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:38:03.324480 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1177/2324] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:38:03.331859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28084557-8dba-43dc-bbfd-ac283f034ac0 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:03.333872 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-28084557-8dba-43dc-bbfd-ac283f034ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:03.407769 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28084557-8dba-43dc-bbfd-ac283f034ac0 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:03.412892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8534ef-03c7-47d2-be67-a0819d565df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:03.413537 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8534ef-03c7-47d2-be67-a0819d565df7 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:03.413828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8534ef-03c7-47d2-be67-a0819d565df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:03.414098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8534ef-03c7-47d2-be67-a0819d565df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:03.425627 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8534ef-03c7-47d2-be67-a0819d565df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:03.427125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8534ef-03c7-47d2-be67-a0819d565df7 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:03.437816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1148/2325] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:38:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:38:09.376486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdf94e-4c76-4240-8b4c-53769d437622 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:09.376486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdf94e-4c76-4240-8b4c-53769d437622 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:09.376486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdf94e-4c76-4240-8b4c-53769d437622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:09.413882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-8ffdf94e-4c76-4240-8b4c-53769d437622 None None] MFA Rules not processed for user `c078e655c9454316aa7afe13a6b565e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:09.415181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8ffdf94e-4c76-4240-8b4c-53769d437622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:09.519647 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1178/2326] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:38:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:38:10.934074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ba365-fe2e-4e30-9691-9a465b95f109 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:10.934856 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ba365-fe2e-4e30-9691-9a465b95f109 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:10.934856 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ba365-fe2e-4e30-9691-9a465b95f109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:10.976706 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-427ba365-fe2e-4e30-9691-9a465b95f109 None None] MFA Rules not processed for user `1a2d7c64ad9d44f997333ace9d4e60a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:10.980036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-427ba365-fe2e-4e30-9691-9a465b95f109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:11.104768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1149/2327] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:38:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:38:11.105623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdf94e-4c76-4240-8b4c-53769d437622 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:11.123873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdf94e-4c76-4240-8b4c-53769d437622 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:11.139552 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c67518-377f-41ee-95e2-4d5c7f8788cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:11.140445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c67518-377f-41ee-95e2-4d5c7f8788cf admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:11.140731 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c67518-377f-41ee-95e2-4d5c7f8788cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:11.141033 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c67518-377f-41ee-95e2-4d5c7f8788cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:11.141694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c67518-377f-41ee-95e2-4d5c7f8788cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:11.143789 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c67518-377f-41ee-95e2-4d5c7f8788cf admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:11.246878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1179/2328] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:38:11 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:38:11.250968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ba365-fe2e-4e30-9691-9a465b95f109 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:11.259887 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:38:11.266344 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ba365-fe2e-4e30-9691-9a465b95f109 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:11.351662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7af21-bb3e-4114-8e04-1bf9437f76bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:11.352681 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7af21-bb3e-4114-8e04-1bf9437f76bd admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:11.353268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7af21-bb3e-4114-8e04-1bf9437f76bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:11.353865 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7af21-bb3e-4114-8e04-1bf9437f76bd admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:11.360054 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7af21-bb3e-4114-8e04-1bf9437f76bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:11.360054 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7af21-bb3e-4114-8e04-1bf9437f76bd admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:11.495102 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1150/2329] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:38:11 2022] POST /identity/v3/users => generated 299 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:38:11.497040 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c67518-377f-41ee-95e2-4d5c7f8788cf admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:11.517913 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c67518-377f-41ee-95e2-4d5c7f8788cf admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:11.527000 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451be8d-bc36-4023-adb5-37fce30e2cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:11.527700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451be8d-bc36-4023-adb5-37fce30e2cae admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:11.528098 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451be8d-bc36-4023-adb5-37fce30e2cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:11.528388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451be8d-bc36-4023-adb5-37fce30e2cae admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:11.529294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451be8d-bc36-4023-adb5-37fce30e2cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:11.531337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451be8d-bc36-4023-adb5-37fce30e2cae admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:11.550296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1180/2330] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:38:11.551687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7af21-bb3e-4114-8e04-1bf9437f76bd admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:11.562961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7af21-bb3e-4114-8e04-1bf9437f76bd admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:11.568212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5438e22-c85b-4c32-8591-3d039f6f44df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:11.568886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5438e22-c85b-4c32-8591-3d039f6f44df admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:11.569145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5438e22-c85b-4c32-8591-3d039f6f44df admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:11.569416 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5438e22-c85b-4c32-8591-3d039f6f44df admin admin] PATH_INFO: `/v3/projects/2e431b83f3a54493a8bbab7af8b2818d/users/ce725aeaefe14291937ee17c4a5654d7/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:11.599796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5438e22-c85b-4c32-8591-3d039f6f44df admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e431b83f3a54493a8bbab7af8b2818d, user_id=ce725aeaefe14291937ee17c4a5654d7, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:11.604454 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5438e22-c85b-4c32-8591-3d039f6f44df admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:11.713816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1151/2331] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:38:11 2022] PUT /identity/v3/projects/2e431b83f3a54493a8bbab7af8b2818d/users/ce725aeaefe14291937ee17c4a5654d7/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:38:11.718619 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451be8d-bc36-4023-adb5-37fce30e2cae admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:11.734452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451be8d-bc36-4023-adb5-37fce30e2cae admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:11.804006 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1864b6c-122d-4a07-a861-39bd5b20d8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:11.804624 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1864b6c-122d-4a07-a861-39bd5b20d8ba admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:11.804909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1864b6c-122d-4a07-a861-39bd5b20d8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:11.805195 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1864b6c-122d-4a07-a861-39bd5b20d8ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:11.805850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1864b6c-122d-4a07-a861-39bd5b20d8ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:11.807518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1864b6c-122d-4a07-a861-39bd5b20d8ba admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:11.823326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1181/2332] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 14:38:11.833015 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5438e22-c85b-4c32-8591-3d039f6f44df admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:11.847548 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5438e22-c85b-4c32-8591-3d039f6f44df admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:11.854474 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01ca96-f13a-4019-8c34-a250393a69cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:11.855694 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01ca96-f13a-4019-8c34-a250393a69cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:11.855987 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01ca96-f13a-4019-8c34-a250393a69cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:11.856438 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01ca96-f13a-4019-8c34-a250393a69cc admin admin] PATH_INFO: `/v3/projects/2e431b83f3a54493a8bbab7af8b2818d/users/ce725aeaefe14291937ee17c4a5654d7/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:11.859556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01ca96-f13a-4019-8c34-a250393a69cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e431b83f3a54493a8bbab7af8b2818d, user_id=ce725aeaefe14291937ee17c4a5654d7, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:11.862197 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01ca96-f13a-4019-8c34-a250393a69cc admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:11.907152 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1152/2333] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:38:11 2022] PUT /identity/v3/projects/2e431b83f3a54493a8bbab7af8b2818d/users/ce725aeaefe14291937ee17c4a5654d7/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:38:19.591778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f40aa5-634d-472c-b809-be27556e1974 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:19.592566 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f40aa5-634d-472c-b809-be27556e1974 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:19.592566 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f40aa5-634d-472c-b809-be27556e1974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:19.632795 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-e4f40aa5-634d-472c-b809-be27556e1974 None None] MFA Rules not processed for user `ce725aeaefe14291937ee17c4a5654d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:19.634205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e4f40aa5-634d-472c-b809-be27556e1974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:19.694382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:38:19.719732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1182/2334] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:38:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:38:19.732809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01ca96-f13a-4019-8c34-a250393a69cc admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:19.734675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8f01ca96-f13a-4019-8c34-a250393a69cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:19.798698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01ca96-f13a-4019-8c34-a250393a69cc admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:19.803111 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3e255-2e9e-4c24-91a0-b35865ade429 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '735986478c354ed196c601f12f3ac377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:19.803800 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3e255-2e9e-4c24-91a0-b35865ade429 service glance] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:19.804079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3e255-2e9e-4c24-91a0-b35865ade429 service glance] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:19.804349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3e255-2e9e-4c24-91a0-b35865ade429 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:19.866924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3e255-2e9e-4c24-91a0-b35865ade429 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:19.868425 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3e255-2e9e-4c24-91a0-b35865ade429 service glance] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:19.878081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1153/2335] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:38:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:38:20.238925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b958f-13bd-4366-a8fa-1a4631dde1f0 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:20.239213 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b958f-13bd-4366-a8fa-1a4631dde1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:20.239491 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b958f-13bd-4366-a8fa-1a4631dde1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:20.259385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-e37b958f-13bd-4366-a8fa-1a4631dde1f0 None None] MFA Rules not processed for user `86300433c724483994f59709a3f015db`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:20.260680 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e37b958f-13bd-4366-a8fa-1a4631dde1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:20.319342 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1183/2336] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 5 14:38:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:38:20.332314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3e255-2e9e-4c24-91a0-b35865ade429 service glance] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:20.333871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-41e3e255-2e9e-4c24-91a0-b35865ade429 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:20.348556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:38:20.356049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a518d-425f-44ed-a93f-831a95f8d169 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:20.356447 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a518d-425f-44ed-a93f-831a95f8d169 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:20.356779 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a518d-425f-44ed-a93f-831a95f8d169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:20.376721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-db3a518d-425f-44ed-a93f-831a95f8d169 None None] MFA Rules not processed for user `86300433c724483994f59709a3f015db`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:20.377932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-db3a518d-425f-44ed-a93f-831a95f8d169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:20.385346 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1184/2337] 10.222.0.15 () {58 vars in 967 bytes} [Mon Sep 5 14:38:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:38:20.398674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a518d-425f-44ed-a93f-831a95f8d169 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:20.406081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3e255-2e9e-4c24-91a0-b35865ade429 service glance] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:20.411067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a518d-425f-44ed-a93f-831a95f8d169 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:20.411724 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef06e5-6448-4d30-99c6-5a9927152b4d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a85809d14cb449d848f22a3f39bf641', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:20.412333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef06e5-6448-4d30-99c6-5a9927152b4d service swift] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:20.412618 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef06e5-6448-4d30-99c6-5a9927152b4d service swift] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:20.412999 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef06e5-6448-4d30-99c6-5a9927152b4d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:20.416397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43664dce-b826-467e-82c1-d439a6099fbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a85809d14cb449d848f22a3f39bf641', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:20.417203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43664dce-b826-467e-82c1-d439a6099fbe service swift] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:20.417483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43664dce-b826-467e-82c1-d439a6099fbe service swift] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:20.417753 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43664dce-b826-467e-82c1-d439a6099fbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:20.423559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef06e5-6448-4d30-99c6-5a9927152b4d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:20.425146 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef06e5-6448-4d30-99c6-5a9927152b4d service swift] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:20.429873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43664dce-b826-467e-82c1-d439a6099fbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:20.431836 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43664dce-b826-467e-82c1-d439a6099fbe service swift] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:20.433723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1154/2338] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 5 14:38:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 05 14:38:20.450601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1185/2339] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 5 14:38:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 05 14:38:24.809143 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef06e5-6448-4d30-99c6-5a9927152b4d service swift] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:24.812128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-20ef06e5-6448-4d30-99c6-5a9927152b4d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:24.932895 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef06e5-6448-4d30-99c6-5a9927152b4d service swift] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:24.938036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70e694-ddc7-4fc2-af6e-7aa5e09c3d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:24.938813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70e694-ddc7-4fc2-af6e-7aa5e09c3d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:24.939224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70e694-ddc7-4fc2-af6e-7aa5e09c3d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:24.939588 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70e694-ddc7-4fc2-af6e-7aa5e09c3d01 admin admin] PATH_INFO: `/v3/users/e8429576e286422cb09591897e82cd69` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:24.940874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70e694-ddc7-4fc2-af6e-7aa5e09c3d01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8429576e286422cb09591897e82cd69)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:24.943065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70e694-ddc7-4fc2-af6e-7aa5e09c3d01 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:25.206394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-5b70e694-ddc7-4fc2-af6e-7aa5e09c3d01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8429576e286422cb09591897e82cd69'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:25.332523 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-5b70e694-ddc7-4fc2-af6e-7aa5e09c3d01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8429576e286422cb09591897e82cd69'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:25.333294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-5b70e694-ddc7-4fc2-af6e-7aa5e09c3d01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8429576e286422cb09591897e82cd69'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:25.349630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-5b70e694-ddc7-4fc2-af6e-7aa5e09c3d01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8429576e286422cb09591897e82cd69'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:25.363161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-5b70e694-ddc7-4fc2-af6e-7aa5e09c3d01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8429576e286422cb09591897e82cd69'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:25.383250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1155/2340] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:38:24 2022] DELETE /identity/v3/users/e8429576e286422cb09591897e82cd69 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:38:25.767210 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43664dce-b826-467e-82c1-d439a6099fbe service swift] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:25.778513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43664dce-b826-467e-82c1-d439a6099fbe service swift] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:25.849432 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93673072-375e-4c1a-8b45-01eaf970d84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:25.850438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93673072-375e-4c1a-8b45-01eaf970d84f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:25.850949 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93673072-375e-4c1a-8b45-01eaf970d84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:25.851287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93673072-375e-4c1a-8b45-01eaf970d84f admin admin] PATH_INFO: `/v3/projects/fc60ee38517541b8bc153807f5332011` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:25.852873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93673072-375e-4c1a-8b45-01eaf970d84f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc60ee38517541b8bc153807f5332011)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:25.854943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93673072-375e-4c1a-8b45-01eaf970d84f admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:25.989425 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-93673072-375e-4c1a-8b45-01eaf970d84f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc60ee38517541b8bc153807f5332011', 'request_id': 'req-93673072-375e-4c1a-8b45-01eaf970d84f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:26.001295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-93673072-375e-4c1a-8b45-01eaf970d84f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc60ee38517541b8bc153807f5332011', 'request_id': 'req-93673072-375e-4c1a-8b45-01eaf970d84f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:26.096432 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-93673072-375e-4c1a-8b45-01eaf970d84f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc60ee38517541b8bc153807f5332011', 'request_id': 'req-93673072-375e-4c1a-8b45-01eaf970d84f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:26.108022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:38:26.112790 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-93673072-375e-4c1a-8b45-01eaf970d84f admin admin] The token cache is being invalidate because project fc60ee38517541b8bc153807f5332011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:38:26.113134 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-93673072-375e-4c1a-8b45-01eaf970d84f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:26.113797 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-93673072-375e-4c1a-8b45-01eaf970d84f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:38:26.121161 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1186/2341] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:38:25 2022] DELETE /identity/v3/projects/fc60ee38517541b8bc153807f5332011 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:38:26.124690 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594ceec-f0bd-45e3-ade9-01f203a2a1fc None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:26.124690 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594ceec-f0bd-45e3-ade9-01f203a2a1fc None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:26.124690 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594ceec-f0bd-45e3-ade9-01f203a2a1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:26.146734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-b594ceec-f0bd-45e3-ade9-01f203a2a1fc None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:26.151558 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b594ceec-f0bd-45e3-ade9-01f203a2a1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:26.174825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:38:26.257605 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1156/2342] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:38:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:38:26.259279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490df31-d618-41d1-9596-0c437f137e0f None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:26.259572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490df31-d618-41d1-9596-0c437f137e0f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:26.259850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490df31-d618-41d1-9596-0c437f137e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:26.287047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-6490df31-d618-41d1-9596-0c437f137e0f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:26.288309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-6490df31-d618-41d1-9596-0c437f137e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:26.297602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1187/2343] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:38:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:38:26.302314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594ceec-f0bd-45e3-ade9-01f203a2a1fc None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:26.317156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594ceec-f0bd-45e3-ade9-01f203a2a1fc None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:26.321888 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe37fdf-0a34-4eca-944a-656a2d4a34e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:26.322667 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe37fdf-0a34-4eca-944a-656a2d4a34e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:26.323209 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe37fdf-0a34-4eca-944a-656a2d4a34e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:26.323891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe37fdf-0a34-4eca-944a-656a2d4a34e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:26.324791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe37fdf-0a34-4eca-944a-656a2d4a34e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:38:26.325531 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe37fdf-0a34-4eca-944a-656a2d4a34e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:26.327867 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe37fdf-0a34-4eca-944a-656a2d4a34e2 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:26.344180 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1157/2344] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:38:26.348493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490df31-d618-41d1-9596-0c437f137e0f None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:26.358738 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490df31-d618-41d1-9596-0c437f137e0f None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:26.365604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7242f06-e056-46a2-b89a-4f6179545f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:26.366422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7242f06-e056-46a2-b89a-4f6179545f9d admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:26.366656 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7242f06-e056-46a2-b89a-4f6179545f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:26.367022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7242f06-e056-46a2-b89a-4f6179545f9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:26.367766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7242f06-e056-46a2-b89a-4f6179545f9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:26.369669 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7242f06-e056-46a2-b89a-4f6179545f9d admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:26.471188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1188/2345] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:38:26 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:38:26.472613 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe37fdf-0a34-4eca-944a-656a2d4a34e2 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:26.486619 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe37fdf-0a34-4eca-944a-656a2d4a34e2 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:26.589746 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebc9dc-f773-4fbe-9185-34f4fefd3743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:26.590377 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebc9dc-f773-4fbe-9185-34f4fefd3743 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:26.590661 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebc9dc-f773-4fbe-9185-34f4fefd3743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:26.590986 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebc9dc-f773-4fbe-9185-34f4fefd3743 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:26.591623 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebc9dc-f773-4fbe-9185-34f4fefd3743 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:26.593957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebc9dc-f773-4fbe-9185-34f4fefd3743 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:26.707708 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1158/2346] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:38:26 2022] POST /identity/v3/users => generated 285 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:38:26.710127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7242f06-e056-46a2-b89a-4f6179545f9d admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:26.725893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7242f06-e056-46a2-b89a-4f6179545f9d admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:26.732783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4651ae-885f-47f7-b741-ff23e84b2103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:26.734414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4651ae-885f-47f7-b741-ff23e84b2103 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:26.734942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4651ae-885f-47f7-b741-ff23e84b2103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:26.735451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4651ae-885f-47f7-b741-ff23e84b2103 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:26.736303 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4651ae-885f-47f7-b741-ff23e84b2103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:26.738581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4651ae-885f-47f7-b741-ff23e84b2103 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:26.763484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1189/2347] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:38:26.763966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebc9dc-f773-4fbe-9185-34f4fefd3743 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:26.782452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebc9dc-f773-4fbe-9185-34f4fefd3743 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:26.785282 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc59a53-8c3f-4002-891d-66909259e5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:26.786757 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc59a53-8c3f-4002-891d-66909259e5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:26.787319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc59a53-8c3f-4002-891d-66909259e5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:26.787755 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc59a53-8c3f-4002-891d-66909259e5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:26.788501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc59a53-8c3f-4002-891d-66909259e5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:26.791052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc59a53-8c3f-4002-891d-66909259e5a8 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:26.814052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1159/2348] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:38:26.818284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4651ae-885f-47f7-b741-ff23e84b2103 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:26.831970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4651ae-885f-47f7-b741-ff23e84b2103 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:26.836997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c3960-09df-4de4-a02f-58136eab9024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:26.838095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c3960-09df-4de4-a02f-58136eab9024 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:26.838376 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c3960-09df-4de4-a02f-58136eab9024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:26.838644 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c3960-09df-4de4-a02f-58136eab9024 admin admin] PATH_INFO: `/v3/projects/172738e7ad624a429835832020a92723/users/60b48954cad94ade83e974fa28063b7d/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:26.898033 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c3960-09df-4de4-a02f-58136eab9024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=172738e7ad624a429835832020a92723, user_id=60b48954cad94ade83e974fa28063b7d, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:26.902727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c3960-09df-4de4-a02f-58136eab9024 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:26.982204 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1190/2349] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:38:26 2022] PUT /identity/v3/projects/172738e7ad624a429835832020a92723/users/60b48954cad94ade83e974fa28063b7d/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:38:26.993708 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa656d-69ba-4aa6-99cc-fcc97b9fcfb5 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:26.994344 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa656d-69ba-4aa6-99cc-fcc97b9fcfb5 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:26.994938 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa656d-69ba-4aa6-99cc-fcc97b9fcfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:27.018285 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-29fa656d-69ba-4aa6-99cc-fcc97b9fcfb5 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:27.019603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-29fa656d-69ba-4aa6-99cc-fcc97b9fcfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:27.114614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1160/2350] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:38:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:38:27.114743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c3960-09df-4de4-a02f-58136eab9024 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:27.115161 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a10c3960-09df-4de4-a02f-58136eab9024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:27.188209 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c3960-09df-4de4-a02f-58136eab9024 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:27.192409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e852b-95b6-42f0-a59f-bdf76a415167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:27.193425 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e852b-95b6-42f0-a59f-bdf76a415167 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:27.193912 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e852b-95b6-42f0-a59f-bdf76a415167 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:27.194402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e852b-95b6-42f0-a59f-bdf76a415167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:27.206722 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e852b-95b6-42f0-a59f-bdf76a415167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:27.208326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e852b-95b6-42f0-a59f-bdf76a415167 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:27.219454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1191/2351] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:38:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:38:34.401377 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0091ff-0e5e-45f2-822a-211b3b174023 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:34.401900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0091ff-0e5e-45f2-822a-211b3b174023 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:34.401994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0091ff-0e5e-45f2-822a-211b3b174023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:34.423401 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-ad0091ff-0e5e-45f2-822a-211b3b174023 None None] MFA Rules not processed for user `1a2d7c64ad9d44f997333ace9d4e60a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:34.424697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ad0091ff-0e5e-45f2-822a-211b3b174023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:34.446655 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:38:34.506532 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1161/2352] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:38:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:38:35.076613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184da61-600f-4e5a-bf0a-7d06858574b3 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:35.078513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184da61-600f-4e5a-bf0a-7d06858574b3 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:35.078995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184da61-600f-4e5a-bf0a-7d06858574b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:35.114338 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-b184da61-600f-4e5a-bf0a-7d06858574b3 None None] MFA Rules not processed for user `60b48954cad94ade83e974fa28063b7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:35.115858 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b184da61-600f-4e5a-bf0a-7d06858574b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:35.201598 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1192/2353] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:38:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:38:35.212548 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0091ff-0e5e-45f2-822a-211b3b174023 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:35.223748 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0091ff-0e5e-45f2-822a-211b3b174023 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:35.228307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda882f-14a6-43f6-a177-adf1ba7997c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:35.228927 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda882f-14a6-43f6-a177-adf1ba7997c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:35.229291 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda882f-14a6-43f6-a177-adf1ba7997c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:35.229483 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda882f-14a6-43f6-a177-adf1ba7997c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:35.230127 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda882f-14a6-43f6-a177-adf1ba7997c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:35.232093 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda882f-14a6-43f6-a177-adf1ba7997c7 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:35.321307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1162/2354] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:38:35 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:38:35.325457 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184da61-600f-4e5a-bf0a-7d06858574b3 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:35.337914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184da61-600f-4e5a-bf0a-7d06858574b3 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:35.369325 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:38:35.424653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ee5dc-a591-4c5f-8156-ee821d8476b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:35.424653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ee5dc-a591-4c5f-8156-ee821d8476b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:35.424973 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ee5dc-a591-4c5f-8156-ee821d8476b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:35.425475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ee5dc-a591-4c5f-8156-ee821d8476b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:35.426224 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ee5dc-a591-4c5f-8156-ee821d8476b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:35.428985 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ee5dc-a591-4c5f-8156-ee821d8476b2 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:35.542303 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1193/2355] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:38:35 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:38:35.549852 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda882f-14a6-43f6-a177-adf1ba7997c7 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:35.564154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda882f-14a6-43f6-a177-adf1ba7997c7 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:35.570769 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27155c-a2f2-4687-b1a3-aa33d3a4d204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:35.571556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27155c-a2f2-4687-b1a3-aa33d3a4d204 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:35.571811 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27155c-a2f2-4687-b1a3-aa33d3a4d204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:35.571965 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27155c-a2f2-4687-b1a3-aa33d3a4d204 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:35.572685 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27155c-a2f2-4687-b1a3-aa33d3a4d204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:35.574289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27155c-a2f2-4687-b1a3-aa33d3a4d204 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:35.590447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1163/2356] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:38:35.591723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ee5dc-a591-4c5f-8156-ee821d8476b2 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:35.603454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ee5dc-a591-4c5f-8156-ee821d8476b2 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:35.611755 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58adebe-8495-4f87-9c84-41f90da335c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:35.612440 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58adebe-8495-4f87-9c84-41f90da335c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:35.619010 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58adebe-8495-4f87-9c84-41f90da335c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:35.619010 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58adebe-8495-4f87-9c84-41f90da335c4 admin admin] PATH_INFO: `/v3/projects/ac15c000034b45ebb9273d817021536d/users/d96837e658544bc7a698718c9feeef38/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:35.632560 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58adebe-8495-4f87-9c84-41f90da335c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac15c000034b45ebb9273d817021536d, user_id=d96837e658544bc7a698718c9feeef38, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:35.635467 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58adebe-8495-4f87-9c84-41f90da335c4 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:35.719874 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1194/2357] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:38:35 2022] PUT /identity/v3/projects/ac15c000034b45ebb9273d817021536d/users/d96837e658544bc7a698718c9feeef38/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:38:35.723960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27155c-a2f2-4687-b1a3-aa33d3a4d204 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:35.734128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27155c-a2f2-4687-b1a3-aa33d3a4d204 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:35.792565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6a0f5-f0d9-4906-89c5-8d2d8ba60705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:35.793171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6a0f5-f0d9-4906-89c5-8d2d8ba60705 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:35.793449 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6a0f5-f0d9-4906-89c5-8d2d8ba60705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:35.793718 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6a0f5-f0d9-4906-89c5-8d2d8ba60705 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:35.794347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6a0f5-f0d9-4906-89c5-8d2d8ba60705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:35.796075 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6a0f5-f0d9-4906-89c5-8d2d8ba60705 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:35.805710 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1164/2358] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:38:35.809960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58adebe-8495-4f87-9c84-41f90da335c4 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:35.820229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58adebe-8495-4f87-9c84-41f90da335c4 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:35.824925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd13104-796b-44f1-9044-b127edb28536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:35.825570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd13104-796b-44f1-9044-b127edb28536 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:35.825850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd13104-796b-44f1-9044-b127edb28536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:35.826150 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd13104-796b-44f1-9044-b127edb28536 admin admin] PATH_INFO: `/v3/projects/ac15c000034b45ebb9273d817021536d/users/d96837e658544bc7a698718c9feeef38/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:35.828583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd13104-796b-44f1-9044-b127edb28536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac15c000034b45ebb9273d817021536d, user_id=d96837e658544bc7a698718c9feeef38, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:35.831234 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd13104-796b-44f1-9044-b127edb28536 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:35.842169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1195/2359] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:38:35 2022] PUT /identity/v3/projects/ac15c000034b45ebb9273d817021536d/users/d96837e658544bc7a698718c9feeef38/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:38:39.693035 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6bb43-972d-4944-94f7-fdcce2a256cc None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:39.693980 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6bb43-972d-4944-94f7-fdcce2a256cc None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:39.694909 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6bb43-972d-4944-94f7-fdcce2a256cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:39.759606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-a5a6bb43-972d-4944-94f7-fdcce2a256cc None None] MFA Rules not processed for user `86300433c724483994f59709a3f015db`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:39.760866 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a5a6bb43-972d-4944-94f7-fdcce2a256cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:39.840230 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1165/2360] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 5 14:38:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:38:39.867090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd13104-796b-44f1-9044-b127edb28536 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:39.868554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7fd13104-796b-44f1-9044-b127edb28536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:39.971082 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd13104-796b-44f1-9044-b127edb28536 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:39.976875 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a308d-1c20-4175-9dca-0b46c05e5876 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a85809d14cb449d848f22a3f39bf641', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:39.977564 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a308d-1c20-4175-9dca-0b46c05e5876 service swift] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:39.977929 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a308d-1c20-4175-9dca-0b46c05e5876 service swift] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:39.978267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a308d-1c20-4175-9dca-0b46c05e5876 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:39.987784 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a308d-1c20-4175-9dca-0b46c05e5876 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:39.990762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a308d-1c20-4175-9dca-0b46c05e5876 service swift] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:40.000889 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1196/2361] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 5 14:38:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 14:38:44.139770 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155ac51-6b0d-4538-a630-ca49583da123 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:44.139770 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155ac51-6b0d-4538-a630-ca49583da123 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:44.139770 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155ac51-6b0d-4538-a630-ca49583da123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:44.190141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-d155ac51-6b0d-4538-a630-ca49583da123 None None] MFA Rules not processed for user `d96837e658544bc7a698718c9feeef38`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:44.191402 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d155ac51-6b0d-4538-a630-ca49583da123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:44.253168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:38:44.313431 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1166/2362] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:38:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:38:44.327879 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a308d-1c20-4175-9dca-0b46c05e5876 service swift] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:38:44.329444 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-192a308d-1c20-4175-9dca-0b46c05e5876 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:44.444849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a308d-1c20-4175-9dca-0b46c05e5876 service swift] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:38:44.451165 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844fdf2-ecf9-402a-8804-1f9df42e2274 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:38:44.452701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844fdf2-ecf9-402a-8804-1f9df42e2274 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:44.453162 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844fdf2-ecf9-402a-8804-1f9df42e2274 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:44.453592 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844fdf2-ecf9-402a-8804-1f9df42e2274 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:44.460024 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:38:44.522454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844fdf2-ecf9-402a-8804-1f9df42e2274 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:38:44.524538 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844fdf2-ecf9-402a-8804-1f9df42e2274 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:38:44.534612 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1197/2363] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:38:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:38:47.944188 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51b1c3-6eee-48ae-afac-a4f88e85db87 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:38:47.944753 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51b1c3-6eee-48ae-afac-a4f88e85db87 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:38:47.944851 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51b1c3-6eee-48ae-afac-a4f88e85db87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:38:47.965770 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-7a51b1c3-6eee-48ae-afac-a4f88e85db87 None None] MFA Rules not processed for user `60b48954cad94ade83e974fa28063b7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:38:47.967006 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-7a51b1c3-6eee-48ae-afac-a4f88e85db87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:38:47.974395 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1167/2364] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:38:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:38:54.592730 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=24 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:38:57.960914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=19 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:39:01.460543 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200c598-9e13-4706-8441-0e3573cf28d2 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:01.461129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200c598-9e13-4706-8441-0e3573cf28d2 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:01.461129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200c598-9e13-4706-8441-0e3573cf28d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:01.483198 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-c200c598-9e13-4706-8441-0e3573cf28d2 None None] MFA Rules not processed for user `86300433c724483994f59709a3f015db`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:01.484443 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c200c598-9e13-4706-8441-0e3573cf28d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:01.492172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1198/2365] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 5 14:39:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:39:01.514021 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51b1c3-6eee-48ae-afac-a4f88e85db87 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:01.527383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51b1c3-6eee-48ae-afac-a4f88e85db87 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:01.533273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0757a2-ef1f-4a6e-ace1-a184cebfa9f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a85809d14cb449d848f22a3f39bf641', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:01.534097 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0757a2-ef1f-4a6e-ace1-a184cebfa9f6 service swift] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:01.534395 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0757a2-ef1f-4a6e-ace1-a184cebfa9f6 service swift] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:01.534678 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0757a2-ef1f-4a6e-ace1-a184cebfa9f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:01.544594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0757a2-ef1f-4a6e-ace1-a184cebfa9f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:01.546237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0757a2-ef1f-4a6e-ace1-a184cebfa9f6 service swift] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:01.555812 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1168/2366] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 5 14:39:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 14:39:02.720683 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200c598-9e13-4706-8441-0e3573cf28d2 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:02.722949 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c200c598-9e13-4706-8441-0e3573cf28d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:02.804476 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200c598-9e13-4706-8441-0e3573cf28d2 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:02.809829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef3dca-4476-45a1-8aa5-635973961c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:02.811276 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef3dca-4476-45a1-8aa5-635973961c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:02.811795 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef3dca-4476-45a1-8aa5-635973961c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:02.812970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef3dca-4476-45a1-8aa5-635973961c30 admin admin] PATH_INFO: `/v3/users/8822d0103d8e40c0961b8cf6c4b4e531` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:02.814129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef3dca-4476-45a1-8aa5-635973961c30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8822d0103d8e40c0961b8cf6c4b4e531)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:02.816097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef3dca-4476-45a1-8aa5-635973961c30 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:03.001588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-feef3dca-4476-45a1-8aa5-635973961c30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8822d0103d8e40c0961b8cf6c4b4e531'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:03.104275 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-feef3dca-4476-45a1-8aa5-635973961c30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8822d0103d8e40c0961b8cf6c4b4e531'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:03.123360 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-feef3dca-4476-45a1-8aa5-635973961c30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8822d0103d8e40c0961b8cf6c4b4e531'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:03.124086 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-feef3dca-4476-45a1-8aa5-635973961c30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8822d0103d8e40c0961b8cf6c4b4e531'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:03.138556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-feef3dca-4476-45a1-8aa5-635973961c30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8822d0103d8e40c0961b8cf6c4b4e531'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:03.150971 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1199/2367] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:39:02 2022] DELETE /identity/v3/users/8822d0103d8e40c0961b8cf6c4b4e531 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:39:03.152296 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32df9f-935a-4bb4-8fc3-40c2ca41978a None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:03.152830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32df9f-935a-4bb4-8fc3-40c2ca41978a None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:03.153027 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32df9f-935a-4bb4-8fc3-40c2ca41978a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:03.191451 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-0f32df9f-935a-4bb4-8fc3-40c2ca41978a None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:03.192970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-0f32df9f-935a-4bb4-8fc3-40c2ca41978a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:03.295619 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1169/2368] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:39:03.307702 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef3dca-4476-45a1-8aa5-635973961c30 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:03.312931 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-feef3dca-4476-45a1-8aa5-635973961c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:03.387827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef3dca-4476-45a1-8aa5-635973961c30 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:03.391991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f3320-b2b8-49f1-8f63-ebe11894b7b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:03.392622 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f3320-b2b8-49f1-8f63-ebe11894b7b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:03.392905 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f3320-b2b8-49f1-8f63-ebe11894b7b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:03.393175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f3320-b2b8-49f1-8f63-ebe11894b7b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:03.404408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f3320-b2b8-49f1-8f63-ebe11894b7b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:03.406049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f3320-b2b8-49f1-8f63-ebe11894b7b4 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:03.416655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1200/2369] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:39:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:39:03.967772 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32df9f-935a-4bb4-8fc3-40c2ca41978a None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:03.969471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-0f32df9f-935a-4bb4-8fc3-40c2ca41978a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:03.991912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32df9f-935a-4bb4-8fc3-40c2ca41978a None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:04.000072 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e06b2e-79fb-4fc4-8c30-df5a8bac7cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:04.000933 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e06b2e-79fb-4fc4-8c30-df5a8bac7cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:04.001679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e06b2e-79fb-4fc4-8c30-df5a8bac7cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:04.002008 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e06b2e-79fb-4fc4-8c30-df5a8bac7cb4 admin admin] PATH_INFO: `/v3/projects/ef79d84f54ff42599908ca6c184805ab` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:04.005001 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e06b2e-79fb-4fc4-8c30-df5a8bac7cb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef79d84f54ff42599908ca6c184805ab)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:04.007803 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e06b2e-79fb-4fc4-8c30-df5a8bac7cb4 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:04.155014 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-93e06b2e-79fb-4fc4-8c30-df5a8bac7cb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef79d84f54ff42599908ca6c184805ab', 'request_id': 'req-93e06b2e-79fb-4fc4-8c30-df5a8bac7cb4'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:04.164923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-93e06b2e-79fb-4fc4-8c30-df5a8bac7cb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef79d84f54ff42599908ca6c184805ab', 'request_id': 'req-93e06b2e-79fb-4fc4-8c30-df5a8bac7cb4'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:04.245029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-93e06b2e-79fb-4fc4-8c30-df5a8bac7cb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef79d84f54ff42599908ca6c184805ab', 'request_id': 'req-93e06b2e-79fb-4fc4-8c30-df5a8bac7cb4'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:04.282513 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-93e06b2e-79fb-4fc4-8c30-df5a8bac7cb4 admin admin] The token cache is being invalidate because project ef79d84f54ff42599908ca6c184805ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:39:04.282932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-93e06b2e-79fb-4fc4-8c30-df5a8bac7cb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:04.287234 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-93e06b2e-79fb-4fc4-8c30-df5a8bac7cb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:04.293730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1170/2370] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:39:03 2022] DELETE /identity/v3/projects/ef79d84f54ff42599908ca6c184805ab => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:39:04.301952 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0f4c2-2d30-4da4-9aca-f5c18de1fde0 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:04.302530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0f4c2-2d30-4da4-9aca-f5c18de1fde0 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:04.304860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0f4c2-2d30-4da4-9aca-f5c18de1fde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:04.393958 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-87b0f4c2-2d30-4da4-9aca-f5c18de1fde0 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:04.395272 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-87b0f4c2-2d30-4da4-9aca-f5c18de1fde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:04.421015 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:39:04.530872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1201/2371] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:39:04.535771 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2043a8a-b564-4066-a0c6-12a3599a754d None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:04.536062 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2043a8a-b564-4066-a0c6-12a3599a754d None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:04.536340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2043a8a-b564-4066-a0c6-12a3599a754d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:04.608824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-f2043a8a-b564-4066-a0c6-12a3599a754d None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:04.610631 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f2043a8a-b564-4066-a0c6-12a3599a754d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:04.628606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0f4c2-2d30-4da4-9aca-f5c18de1fde0 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:04.629206 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1171/2372] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:39:04.649659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0f4c2-2d30-4da4-9aca-f5c18de1fde0 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:04.663775 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7daeb-a34d-410f-bad6-bc9478a3e906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:04.664566 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7daeb-a34d-410f-bad6-bc9478a3e906 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:04.664690 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7daeb-a34d-410f-bad6-bc9478a3e906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:04.664945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7daeb-a34d-410f-bad6-bc9478a3e906 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:04.665806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7daeb-a34d-410f-bad6-bc9478a3e906 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:39:04.666117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7daeb-a34d-410f-bad6-bc9478a3e906 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:04.667884 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7daeb-a34d-410f-bad6-bc9478a3e906 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:04.692121 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1202/2373] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 14:39:04.696061 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2043a8a-b564-4066-a0c6-12a3599a754d None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:04.721657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2043a8a-b564-4066-a0c6-12a3599a754d None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:04.738453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ee851-3196-49e7-961a-3f5b4adb3cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:04.739527 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ee851-3196-49e7-961a-3f5b4adb3cfc admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:04.739527 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ee851-3196-49e7-961a-3f5b4adb3cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:04.739783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ee851-3196-49e7-961a-3f5b4adb3cfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:04.740389 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ee851-3196-49e7-961a-3f5b4adb3cfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:04.742387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ee851-3196-49e7-961a-3f5b4adb3cfc admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:04.994110 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1172/2374] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:39:04 2022] POST /identity/v3/projects => generated 373 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:39:04.998254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7daeb-a34d-410f-bad6-bc9478a3e906 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:05.009919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7daeb-a34d-410f-bad6-bc9478a3e906 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:05.066946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740adb1-9ba9-426e-8a59-0c50b94501a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:05.067568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740adb1-9ba9-426e-8a59-0c50b94501a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:05.067847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740adb1-9ba9-426e-8a59-0c50b94501a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:05.068116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740adb1-9ba9-426e-8a59-0c50b94501a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:05.068762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9740adb1-9ba9-426e-8a59-0c50b94501a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:05.070825 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9740adb1-9ba9-426e-8a59-0c50b94501a5 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:05.256089 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1203/2375] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:39:04 2022] POST /identity/v3/users => generated 287 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:39:05.260581 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ee851-3196-49e7-961a-3f5b4adb3cfc admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:05.271605 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ee851-3196-49e7-961a-3f5b4adb3cfc admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:05.276909 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccec1d6-68d3-4595-bc13-b055b7b53936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:05.277534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccec1d6-68d3-4595-bc13-b055b7b53936 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:05.277817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccec1d6-68d3-4595-bc13-b055b7b53936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:05.278096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccec1d6-68d3-4595-bc13-b055b7b53936 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:05.278782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccec1d6-68d3-4595-bc13-b055b7b53936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:05.282084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccec1d6-68d3-4595-bc13-b055b7b53936 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:05.291475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1173/2376] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:39:05.295735 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740adb1-9ba9-426e-8a59-0c50b94501a5 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:05.306430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740adb1-9ba9-426e-8a59-0c50b94501a5 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:05.311071 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80194d-c4a5-440d-ac76-4c5f086a173f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:05.311664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80194d-c4a5-440d-ac76-4c5f086a173f admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:05.311945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80194d-c4a5-440d-ac76-4c5f086a173f admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:05.312220 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80194d-c4a5-440d-ac76-4c5f086a173f admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:05.312858 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80194d-c4a5-440d-ac76-4c5f086a173f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:05.314456 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80194d-c4a5-440d-ac76-4c5f086a173f admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:05.324968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1204/2377] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:39:05.329302 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccec1d6-68d3-4595-bc13-b055b7b53936 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:05.340147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccec1d6-68d3-4595-bc13-b055b7b53936 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:05.345623 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc15878-6d38-42f5-836e-a3feb91968c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:05.346282 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc15878-6d38-42f5-836e-a3feb91968c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:05.346565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc15878-6d38-42f5-836e-a3feb91968c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:05.346873 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc15878-6d38-42f5-836e-a3feb91968c0 admin admin] PATH_INFO: `/v3/projects/56deac6e76ee41f38fd2ca1f5a6536ec/users/53398ecdd31e447ea317eeeaa78a0927/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:05.362491 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc15878-6d38-42f5-836e-a3feb91968c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56deac6e76ee41f38fd2ca1f5a6536ec, user_id=53398ecdd31e447ea317eeeaa78a0927, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:05.365178 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc15878-6d38-42f5-836e-a3feb91968c0 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:05.439430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1174/2378] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:39:05 2022] PUT /identity/v3/projects/56deac6e76ee41f38fd2ca1f5a6536ec/users/53398ecdd31e447ea317eeeaa78a0927/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:39:05.449867 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfd4f3-201e-4977-9c94-413efb958f6e None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:05.450158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfd4f3-201e-4977-9c94-413efb958f6e None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:05.452359 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfd4f3-201e-4977-9c94-413efb958f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:05.483505 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-05bfd4f3-201e-4977-9c94-413efb958f6e None None] MFA Rules not processed for user `53398ecdd31e447ea317eeeaa78a0927`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:05.485622 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-05bfd4f3-201e-4977-9c94-413efb958f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:05.574843 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1205/2379] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:39:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:39:05.588297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc15878-6d38-42f5-836e-a3feb91968c0 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:05.590348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-edc15878-6d38-42f5-836e-a3feb91968c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:05.605252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:39:05.667536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc15878-6d38-42f5-836e-a3feb91968c0 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:05.671721 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1136a3-583d-4345-8474-f93ac190713e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:05.678545 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1136a3-583d-4345-8474-f93ac190713e service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:05.678545 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1136a3-583d-4345-8474-f93ac190713e service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:05.678545 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1136a3-583d-4345-8474-f93ac190713e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:05.684009 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1136a3-583d-4345-8474-f93ac190713e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:05.685460 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1136a3-583d-4345-8474-f93ac190713e service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:05.701675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1175/2380] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:39:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:39:08.788675 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbda62-0af9-4498-bfe6-f1c3c9118c41 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:08.790876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbda62-0af9-4498-bfe6-f1c3c9118c41 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:08.791311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbda62-0af9-4498-bfe6-f1c3c9118c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:08.876095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-47cbda62-0af9-4498-bfe6-f1c3c9118c41 None None] MFA Rules not processed for user `53398ecdd31e447ea317eeeaa78a0927`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:08.877369 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-47cbda62-0af9-4498-bfe6-f1c3c9118c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:08.891245 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1206/2381] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:39:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:39:11.488978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1136a3-583d-4345-8474-f93ac190713e service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:11.491223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8f1136a3-583d-4345-8474-f93ac190713e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:11.555064 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1136a3-583d-4345-8474-f93ac190713e service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:11.559270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804c42e-d512-48a2-a2b6-5f2b4723b2c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:11.560086 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804c42e-d512-48a2-a2b6-5f2b4723b2c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:11.560370 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804c42e-d512-48a2-a2b6-5f2b4723b2c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:11.560651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804c42e-d512-48a2-a2b6-5f2b4723b2c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:11.562615 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e804c42e-d512-48a2-a2b6-5f2b4723b2c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:11.651009 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804c42e-d512-48a2-a2b6-5f2b4723b2c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:11.653968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804c42e-d512-48a2-a2b6-5f2b4723b2c2 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:11.671137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1176/2382] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:39:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:39:13.089042 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbda62-0af9-4498-bfe6-f1c3c9118c41 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:13.104740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:39:13.115200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbda62-0af9-4498-bfe6-f1c3c9118c41 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:13.147381 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58735979-754c-4805-9720-cc6e0aeb9b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:13.152239 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58735979-754c-4805-9720-cc6e0aeb9b0b service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:13.152239 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58735979-754c-4805-9720-cc6e0aeb9b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:13.152239 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58735979-754c-4805-9720-cc6e0aeb9b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:13.156426 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-58735979-754c-4805-9720-cc6e0aeb9b0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:13.193243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58735979-754c-4805-9720-cc6e0aeb9b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:13.194766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58735979-754c-4805-9720-cc6e0aeb9b0b service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:13.215653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1207/2383] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:39:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:39:21.662954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=77 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:39:23.210474 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:39:24.674506 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804c42e-d512-48a2-a2b6-5f2b4723b2c2 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:24.683772 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804c42e-d512-48a2-a2b6-5f2b4723b2c2 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:24.687954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180d57e-cae9-4a6f-8d29-cbbb69ec5e52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:24.688583 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180d57e-cae9-4a6f-8d29-cbbb69ec5e52 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:24.688869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180d57e-cae9-4a6f-8d29-cbbb69ec5e52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:24.689150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180d57e-cae9-4a6f-8d29-cbbb69ec5e52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:24.690970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-3180d57e-cae9-4a6f-8d29-cbbb69ec5e52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:24.755909 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180d57e-cae9-4a6f-8d29-cbbb69ec5e52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:24.757357 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180d57e-cae9-4a6f-8d29-cbbb69ec5e52 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:24.767891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1177/2384] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:39:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:39:26.181734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58735979-754c-4805-9720-cc6e0aeb9b0b service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:26.186695 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-58735979-754c-4805-9720-cc6e0aeb9b0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:26.200328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58735979-754c-4805-9720-cc6e0aeb9b0b service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:26.212016 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc58c97-8710-4389-8bac-cb1778916f61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:26.217856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc58c97-8710-4389-8bac-cb1778916f61 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:26.217856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc58c97-8710-4389-8bac-cb1778916f61 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:26.217856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc58c97-8710-4389-8bac-cb1778916f61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:26.217856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-dcc58c97-8710-4389-8bac-cb1778916f61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:26.278861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc58c97-8710-4389-8bac-cb1778916f61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:26.280391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc58c97-8710-4389-8bac-cb1778916f61 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:26.289557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1208/2385] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:39:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:39:28.389936 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a844bc9-3b3f-4159-90f6-5427557f54b8 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:28.390750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a844bc9-3b3f-4159-90f6-5427557f54b8 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:28.391184 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a844bc9-3b3f-4159-90f6-5427557f54b8 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:28.392788 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1178/2386] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 5 14:39:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:39:28.399140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6a955-6b75-4e81-89a2-05238c22268c None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:28.399632 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6a955-6b75-4e81-89a2-05238c22268c None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:28.399718 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6a955-6b75-4e81-89a2-05238c22268c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:28.426848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-12a6a955-6b75-4e81-89a2-05238c22268c None None] MFA Rules not processed for user `709b9ae82dcd47408fc9a6d54a54428a`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:28.428387 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-12a6a955-6b75-4e81-89a2-05238c22268c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:28.443830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1209/2387] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 5 14:39:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:39:28.453298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a844bc9-3b3f-4159-90f6-5427557f54b8 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:28.455145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8a844bc9-3b3f-4159-90f6-5427557f54b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:28.558386 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a844bc9-3b3f-4159-90f6-5427557f54b8 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:28.563074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d739d9-29f6-4c80-9895-e2db0a5cecf9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '735986478c354ed196c601f12f3ac377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:28.563703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d739d9-29f6-4c80-9895-e2db0a5cecf9 service glance] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:28.563981 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d739d9-29f6-4c80-9895-e2db0a5cecf9 service glance] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:28.564256 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d739d9-29f6-4c80-9895-e2db0a5cecf9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:28.577410 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d739d9-29f6-4c80-9895-e2db0a5cecf9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:28.578893 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d739d9-29f6-4c80-9895-e2db0a5cecf9 service glance] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:28.594481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1179/2388] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:39:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:39:28.845059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f62f6-e020-4d77-86b1-e1a5fbc6bd33 None None] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:28.845346 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f62f6-e020-4d77-86b1-e1a5fbc6bd33 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:28.846192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f62f6-e020-4d77-86b1-e1a5fbc6bd33 None None] PATH_INFO: `/` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:28.847166 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1210/2389] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 5 14:39:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:39:28.853203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdaf909-8016-49d7-9792-548176001c03 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:28.853624 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdaf909-8016-49d7-9792-548176001c03 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:28.854114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdaf909-8016-49d7-9792-548176001c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:28.890720 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-9bdaf909-8016-49d7-9792-548176001c03 None None] MFA Rules not processed for user `5c73ca2785334fc496b5227d3be655c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:28.892976 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9bdaf909-8016-49d7-9792-548176001c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:28.903604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1180/2390] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 5 14:39:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:39:28.934761 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f62f6-e020-4d77-86b1-e1a5fbc6bd33 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:28.950664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f62f6-e020-4d77-86b1-e1a5fbc6bd33 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:28.956249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153060a-a1e2-40bf-9d1c-8f5d9d120ba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:28.959181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153060a-a1e2-40bf-9d1c-8f5d9d120ba3 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:28.959574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153060a-a1e2-40bf-9d1c-8f5d9d120ba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:28.960246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153060a-a1e2-40bf-9d1c-8f5d9d120ba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:28.968361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153060a-a1e2-40bf-9d1c-8f5d9d120ba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:28.970126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153060a-a1e2-40bf-9d1c-8f5d9d120ba3 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:28.978581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1211/2391] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:39:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:39:29.131101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddad463-1fdc-482e-b86d-2a2a1c8bf149 None None] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:29.131657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddad463-1fdc-482e-b86d-2a2a1c8bf149 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:29.132621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddad463-1fdc-482e-b86d-2a2a1c8bf149 None None] PATH_INFO: `/` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:29.134698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1181/2392] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 5 14:39:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 14:39:29.140673 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb01fc-9b29-4143-89bd-d107ffc44a0d None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:29.141097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb01fc-9b29-4143-89bd-d107ffc44a0d None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:29.146265 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb01fc-9b29-4143-89bd-d107ffc44a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:29.164069 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-0afb01fc-9b29-4143-89bd-d107ffc44a0d None None] MFA Rules not processed for user `709b9ae82dcd47408fc9a6d54a54428a`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:29.165276 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0afb01fc-9b29-4143-89bd-d107ffc44a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:29.176747 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1212/2393] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 5 14:39:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:39:29.192369 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddad463-1fdc-482e-b86d-2a2a1c8bf149 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:29.192369 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-fddad463-1fdc-482e-b86d-2a2a1c8bf149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:29.204432 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddad463-1fdc-482e-b86d-2a2a1c8bf149 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:29.208490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ee97c-ffd1-4806-8289-7ddf36b3de97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:29.209179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ee97c-ffd1-4806-8289-7ddf36b3de97 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:29.209522 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ee97c-ffd1-4806-8289-7ddf36b3de97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:29.209864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ee97c-ffd1-4806-8289-7ddf36b3de97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:29.221428 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ee97c-ffd1-4806-8289-7ddf36b3de97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:29.223022 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ee97c-ffd1-4806-8289-7ddf36b3de97 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:29.239285 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1182/2394] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:39:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:39:29.309038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb01fc-9b29-4143-89bd-d107ffc44a0d None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:29.310539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0afb01fc-9b29-4143-89bd-d107ffc44a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:29.390731 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb01fc-9b29-4143-89bd-d107ffc44a0d None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:29.396766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600eca5-acbd-4423-a04a-6d94f15a065c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:29.397602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600eca5-acbd-4423-a04a-6d94f15a065c service placement] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:29.398434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600eca5-acbd-4423-a04a-6d94f15a065c service placement] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:29.398921 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600eca5-acbd-4423-a04a-6d94f15a065c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:29.401125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d600eca5-acbd-4423-a04a-6d94f15a065c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:29.418906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600eca5-acbd-4423-a04a-6d94f15a065c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:29.421392 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600eca5-acbd-4423-a04a-6d94f15a065c service placement] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:29.440252 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1213/2395] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:39:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:39:31.228763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ee97c-ffd1-4806-8289-7ddf36b3de97 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:31.241289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ee97c-ffd1-4806-8289-7ddf36b3de97 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:31.251127 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68306df0-46b3-4369-8199-ec24d2ef51ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:31.251747 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68306df0-46b3-4369-8199-ec24d2ef51ec service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:31.254853 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68306df0-46b3-4369-8199-ec24d2ef51ec service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:31.255228 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68306df0-46b3-4369-8199-ec24d2ef51ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:31.257530 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-68306df0-46b3-4369-8199-ec24d2ef51ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:31.357361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68306df0-46b3-4369-8199-ec24d2ef51ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:31.360215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68306df0-46b3-4369-8199-ec24d2ef51ec service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:31.373376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1183/2396] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:39:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:39:37.541049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeb5e8-86a9-4b77-af6b-37557329da13 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:37.541584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeb5e8-86a9-4b77-af6b-37557329da13 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:37.541584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeb5e8-86a9-4b77-af6b-37557329da13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:37.563100 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-defeb5e8-86a9-4b77-af6b-37557329da13 None None] MFA Rules not processed for user `86300433c724483994f59709a3f015db`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:37.564321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-defeb5e8-86a9-4b77-af6b-37557329da13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:37.637335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1214/2397] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 5 14:39:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:39:37.647536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68306df0-46b3-4369-8199-ec24d2ef51ec service nova] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:37.649153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-68306df0-46b3-4369-8199-ec24d2ef51ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:37.669920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=100 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:39:37.754025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68306df0-46b3-4369-8199-ec24d2ef51ec service nova] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:37.759324 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac4e37-e77a-42b0-aefd-3aa619c77b6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a85809d14cb449d848f22a3f39bf641', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:37.759961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac4e37-e77a-42b0-aefd-3aa619c77b6e service swift] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:37.760386 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac4e37-e77a-42b0-aefd-3aa619c77b6e service swift] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:37.760664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac4e37-e77a-42b0-aefd-3aa619c77b6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:37.769711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac4e37-e77a-42b0-aefd-3aa619c77b6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:37.771579 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac4e37-e77a-42b0-aefd-3aa619c77b6e service swift] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:37.782683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1184/2398] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 5 14:39:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 14:39:38.263339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517fe4f-ff25-4243-9bdd-d5146c3d437c None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:38.264948 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517fe4f-ff25-4243-9bdd-d5146c3d437c None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:38.264948 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517fe4f-ff25-4243-9bdd-d5146c3d437c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:38.277704 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:39:38.284330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-4517fe4f-ff25-4243-9bdd-d5146c3d437c None None] MFA Rules not processed for user `86300433c724483994f59709a3f015db`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:38.285545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-4517fe4f-ff25-4243-9bdd-d5146c3d437c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:38.292932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1215/2399] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 5 14:39:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:39:38.305787 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac4e37-e77a-42b0-aefd-3aa619c77b6e service swift] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:38.316178 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac4e37-e77a-42b0-aefd-3aa619c77b6e service swift] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:38.323915 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d438b-90c0-4a08-82e5-7759d746372f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a85809d14cb449d848f22a3f39bf641', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:38.323915 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d438b-90c0-4a08-82e5-7759d746372f service swift] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:38.323915 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d438b-90c0-4a08-82e5-7759d746372f service swift] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:38.323915 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d438b-90c0-4a08-82e5-7759d746372f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:38.331812 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d438b-90c0-4a08-82e5-7759d746372f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:38.333380 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d438b-90c0-4a08-82e5-7759d746372f service swift] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:38.341914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1185/2400] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 5 14:39:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 14:39:38.765578 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63fcbe-be3e-4c50-8c2a-eb199563fa38 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:38.766022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63fcbe-be3e-4c50-8c2a-eb199563fa38 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:38.766022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63fcbe-be3e-4c50-8c2a-eb199563fa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:38.807969 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-9c63fcbe-be3e-4c50-8c2a-eb199563fa38 None None] MFA Rules not processed for user `86300433c724483994f59709a3f015db`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:38.809569 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-9c63fcbe-be3e-4c50-8c2a-eb199563fa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:38.817300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1216/2401] 10.222.0.15 () {58 vars in 967 bytes} [Mon Sep 5 14:39:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:39:38.830237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d438b-90c0-4a08-82e5-7759d746372f service swift] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:38.846441 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d438b-90c0-4a08-82e5-7759d746372f service swift] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:38.858289 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcfd70-55bb-4115-84e6-1f7a15938e2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a85809d14cb449d848f22a3f39bf641', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:38.859105 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcfd70-55bb-4115-84e6-1f7a15938e2a service swift] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:38.859454 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcfd70-55bb-4115-84e6-1f7a15938e2a service swift] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:38.859803 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcfd70-55bb-4115-84e6-1f7a15938e2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:38.869765 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bcfd70-55bb-4115-84e6-1f7a15938e2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:38.872226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bcfd70-55bb-4115-84e6-1f7a15938e2a service swift] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:38.883485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1186/2402] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 5 14:39:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 14:39:39.719233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24929e-5700-402d-a668-d0ba008887f5 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:39.719517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24929e-5700-402d-a668-d0ba008887f5 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:39.719793 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24929e-5700-402d-a668-d0ba008887f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:39.741042 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-3d24929e-5700-402d-a668-d0ba008887f5 None None] MFA Rules not processed for user `86300433c724483994f59709a3f015db`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:39.742370 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-3d24929e-5700-402d-a668-d0ba008887f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:39.751673 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1217/2403] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 5 14:39:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:39:39.763413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcfd70-55bb-4115-84e6-1f7a15938e2a service swift] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:39.774594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcfd70-55bb-4115-84e6-1f7a15938e2a service swift] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:39.779812 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd089fa2-c5ad-4f69-9327-31dfb9768a99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a85809d14cb449d848f22a3f39bf641', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:39.780494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd089fa2-c5ad-4f69-9327-31dfb9768a99 service swift] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:39.780874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd089fa2-c5ad-4f69-9327-31dfb9768a99 service swift] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:39.781087 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd089fa2-c5ad-4f69-9327-31dfb9768a99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:39.789574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd089fa2-c5ad-4f69-9327-31dfb9768a99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:39.791255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd089fa2-c5ad-4f69-9327-31dfb9768a99 service swift] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:39.800032 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1187/2404] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 5 14:39:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 14:39:40.709513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047fa93-8962-4461-af3c-59e891a89a14 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:40.710025 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047fa93-8962-4461-af3c-59e891a89a14 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:40.710131 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047fa93-8962-4461-af3c-59e891a89a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:40.746560 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-0047fa93-8962-4461-af3c-59e891a89a14 None None] MFA Rules not processed for user `86300433c724483994f59709a3f015db`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:40.749929 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0047fa93-8962-4461-af3c-59e891a89a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:40.758752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1218/2405] 10.222.0.15 () {58 vars in 967 bytes} [Mon Sep 5 14:39:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:39:40.781479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd089fa2-c5ad-4f69-9327-31dfb9768a99 service swift] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:40.793332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd089fa2-c5ad-4f69-9327-31dfb9768a99 service swift] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:40.799669 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bbea2-6b8a-48ab-86d5-0f39d01118a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a85809d14cb449d848f22a3f39bf641', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:40.800466 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bbea2-6b8a-48ab-86d5-0f39d01118a7 service swift] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:40.800943 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bbea2-6b8a-48ab-86d5-0f39d01118a7 service swift] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:40.801555 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bbea2-6b8a-48ab-86d5-0f39d01118a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:40.814440 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8bbea2-6b8a-48ab-86d5-0f39d01118a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:40.816239 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8bbea2-6b8a-48ab-86d5-0f39d01118a7 service swift] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:40.825845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1188/2406] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 5 14:39:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 14:39:43.717395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047fa93-8962-4461-af3c-59e891a89a14 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:43.788653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047fa93-8962-4461-af3c-59e891a89a14 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:43.793469 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd4799-ed46-4411-97d5-7aa2d13002b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:43.794117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd4799-ed46-4411-97d5-7aa2d13002b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:43.794391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd4799-ed46-4411-97d5-7aa2d13002b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:43.794665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd4799-ed46-4411-97d5-7aa2d13002b1 admin admin] PATH_INFO: `/v3/users/53398ecdd31e447ea317eeeaa78a0927` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:43.795840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd4799-ed46-4411-97d5-7aa2d13002b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53398ecdd31e447ea317eeeaa78a0927)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:43.797806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd4799-ed46-4411-97d5-7aa2d13002b1 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:44.035487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d9bd4799-ed46-4411-97d5-7aa2d13002b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53398ecdd31e447ea317eeeaa78a0927'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:44.176009 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d9bd4799-ed46-4411-97d5-7aa2d13002b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53398ecdd31e447ea317eeeaa78a0927'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:44.194861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d9bd4799-ed46-4411-97d5-7aa2d13002b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53398ecdd31e447ea317eeeaa78a0927'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:44.195600 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d9bd4799-ed46-4411-97d5-7aa2d13002b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53398ecdd31e447ea317eeeaa78a0927'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:44.202461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-d9bd4799-ed46-4411-97d5-7aa2d13002b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53398ecdd31e447ea317eeeaa78a0927'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:44.208426 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1219/2407] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:39:43 2022] DELETE /identity/v3/users/53398ecdd31e447ea317eeeaa78a0927 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:39:44.212887 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395a103-b218-4257-a65d-e2770970f778 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:44.213413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395a103-b218-4257-a65d-e2770970f778 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:44.213413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395a103-b218-4257-a65d-e2770970f778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:44.236659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-0395a103-b218-4257-a65d-e2770970f778 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:44.238536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-0395a103-b218-4257-a65d-e2770970f778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:44.318927 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1189/2408] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:39:44.326637 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd4799-ed46-4411-97d5-7aa2d13002b1 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:44.328202 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d9bd4799-ed46-4411-97d5-7aa2d13002b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:44.391434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd4799-ed46-4411-97d5-7aa2d13002b1 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:44.395199 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543dfa9-bbaa-4b40-816d-9e5dfcb9bfc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:44.395813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543dfa9-bbaa-4b40-816d-9e5dfcb9bfc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:44.396094 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543dfa9-bbaa-4b40-816d-9e5dfcb9bfc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:44.396383 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543dfa9-bbaa-4b40-816d-9e5dfcb9bfc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:44.415623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543dfa9-bbaa-4b40-816d-9e5dfcb9bfc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:44.417059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543dfa9-bbaa-4b40-816d-9e5dfcb9bfc2 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:44.428139 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1220/2409] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:39:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:39:44.716833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395a103-b218-4257-a65d-e2770970f778 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:44.719345 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-0395a103-b218-4257-a65d-e2770970f778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:44.738850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395a103-b218-4257-a65d-e2770970f778 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:44.743640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775aa29-8454-46c3-a001-77326fdfa435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:44.744341 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775aa29-8454-46c3-a001-77326fdfa435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:44.744632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775aa29-8454-46c3-a001-77326fdfa435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:44.744929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775aa29-8454-46c3-a001-77326fdfa435 admin admin] PATH_INFO: `/v3/projects/56deac6e76ee41f38fd2ca1f5a6536ec` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:44.746150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775aa29-8454-46c3-a001-77326fdfa435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56deac6e76ee41f38fd2ca1f5a6536ec)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:44.752960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775aa29-8454-46c3-a001-77326fdfa435 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:44.837983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:39:44.840131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-d775aa29-8454-46c3-a001-77326fdfa435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56deac6e76ee41f38fd2ca1f5a6536ec', 'request_id': 'req-d775aa29-8454-46c3-a001-77326fdfa435'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:44.849453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-d775aa29-8454-46c3-a001-77326fdfa435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56deac6e76ee41f38fd2ca1f5a6536ec', 'request_id': 'req-d775aa29-8454-46c3-a001-77326fdfa435'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:44.946799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-d775aa29-8454-46c3-a001-77326fdfa435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56deac6e76ee41f38fd2ca1f5a6536ec', 'request_id': 'req-d775aa29-8454-46c3-a001-77326fdfa435'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:44.960117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-d775aa29-8454-46c3-a001-77326fdfa435 admin admin] The token cache is being invalidate because project 56deac6e76ee41f38fd2ca1f5a6536ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:39:44.960297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-d775aa29-8454-46c3-a001-77326fdfa435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:44.960971 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-d775aa29-8454-46c3-a001-77326fdfa435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:39:44.962501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1190/2410] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:39:44 2022] DELETE /identity/v3/projects/56deac6e76ee41f38fd2ca1f5a6536ec => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:39:44.967310 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa80a9-a302-488d-a0d1-4750646e4dd9 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:44.968786 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa80a9-a302-488d-a0d1-4750646e4dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:44.968786 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa80a9-a302-488d-a0d1-4750646e4dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:44.979743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:39:44.992889 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-a2fa80a9-a302-488d-a0d1-4750646e4dd9 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:44.994219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a2fa80a9-a302-488d-a0d1-4750646e4dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:45.057951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1221/2411] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:39:45.065569 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75233f7-7fb4-4159-8ed2-08724ff2031d None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:45.066175 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75233f7-7fb4-4159-8ed2-08724ff2031d None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:45.066523 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75233f7-7fb4-4159-8ed2-08724ff2031d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:45.091321 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-b75233f7-7fb4-4159-8ed2-08724ff2031d None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:45.092571 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b75233f7-7fb4-4159-8ed2-08724ff2031d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:45.101333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1191/2412] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 14:39:45.105665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa80a9-a302-488d-a0d1-4750646e4dd9 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:45.116189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa80a9-a302-488d-a0d1-4750646e4dd9 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:45.120805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0ea0d-1962-4e8c-a852-a9050e7f03c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:45.121421 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0ea0d-1962-4e8c-a852-a9050e7f03c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:45.121698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0ea0d-1962-4e8c-a852-a9050e7f03c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:45.121968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0ea0d-1962-4e8c-a852-a9050e7f03c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:45.122620 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0ea0d-1962-4e8c-a852-a9050e7f03c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 14:39:45.122999 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0ea0d-1962-4e8c-a852-a9050e7f03c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:45.124666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0ea0d-1962-4e8c-a852-a9050e7f03c1 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:45.137179 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1222/2413] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 14:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 14:39:45.141967 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75233f7-7fb4-4159-8ed2-08724ff2031d None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:45.151330 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75233f7-7fb4-4159-8ed2-08724ff2031d None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:45.156865 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dface63-63b3-4a43-a0e9-a28850767587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:45.157475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dface63-63b3-4a43-a0e9-a28850767587 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:45.157753 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dface63-63b3-4a43-a0e9-a28850767587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:45.158023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dface63-63b3-4a43-a0e9-a28850767587 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:45.158654 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dface63-63b3-4a43-a0e9-a28850767587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:45.160603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dface63-63b3-4a43-a0e9-a28850767587 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:45.251450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1192/2414] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 14:39:45 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 14:39:45.255794 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0ea0d-1962-4e8c-a852-a9050e7f03c1 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:45.267625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0ea0d-1962-4e8c-a852-a9050e7f03c1 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:45.330352 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f2918-eda5-42f9-ab5b-768d61e9c237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:45.331438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f2918-eda5-42f9-ab5b-768d61e9c237 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:45.331916 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f2918-eda5-42f9-ab5b-768d61e9c237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:45.332353 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f2918-eda5-42f9-ab5b-768d61e9c237 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:45.333152 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f2918-eda5-42f9-ab5b-768d61e9c237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:45.335262 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f2918-eda5-42f9-ab5b-768d61e9c237 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:45.433186 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1223/2415] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 14:39:45 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 14:39:45.436969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dface63-63b3-4a43-a0e9-a28850767587 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:45.449696 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dface63-63b3-4a43-a0e9-a28850767587 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:45.455310 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb62083-7556-46a6-b050-50aa273d5ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:45.456011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb62083-7556-46a6-b050-50aa273d5ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:45.456364 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb62083-7556-46a6-b050-50aa273d5ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:45.457341 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb62083-7556-46a6-b050-50aa273d5ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:45.458061 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb62083-7556-46a6-b050-50aa273d5ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:45.459878 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb62083-7556-46a6-b050-50aa273d5ca5 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:45.601928 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1193/2416] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:39:45.603129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f2918-eda5-42f9-ab5b-768d61e9c237 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:45.617733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f2918-eda5-42f9-ab5b-768d61e9c237 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:45.628409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a4588-229d-42dc-9a05-380fb5bc783b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:45.628409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a4588-229d-42dc-9a05-380fb5bc783b admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:45.628409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a4588-229d-42dc-9a05-380fb5bc783b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:45.628409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a4588-229d-42dc-9a05-380fb5bc783b admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:45.636209 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a4588-229d-42dc-9a05-380fb5bc783b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:45.638465 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a4588-229d-42dc-9a05-380fb5bc783b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:45.666849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1224/2417] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 14:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 14:39:45.669323 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb62083-7556-46a6-b050-50aa273d5ca5 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:45.681378 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb62083-7556-46a6-b050-50aa273d5ca5 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:45.693114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef67ad-3d5e-45e3-8388-9e2f6240145d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:45.696795 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef67ad-3d5e-45e3-8388-9e2f6240145d admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:45.696795 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef67ad-3d5e-45e3-8388-9e2f6240145d admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:45.696795 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef67ad-3d5e-45e3-8388-9e2f6240145d admin admin] PATH_INFO: `/v3/projects/f60d54b7dc874a8080159b81aafc89d5/users/dac33ec33c26411d9dcf8c3bbea7a9e9/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:45.711665 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef67ad-3d5e-45e3-8388-9e2f6240145d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f60d54b7dc874a8080159b81aafc89d5, user_id=dac33ec33c26411d9dcf8c3bbea7a9e9, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:45.714283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef67ad-3d5e-45e3-8388-9e2f6240145d admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:45.819649 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1194/2418] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 14:39:45 2022] PUT /identity/v3/projects/f60d54b7dc874a8080159b81aafc89d5/users/dac33ec33c26411d9dcf8c3bbea7a9e9/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 14:39:45.824780 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a915676-e7ea-4cd0-a1a5-0c3255d31c50 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:45.825071 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a915676-e7ea-4cd0-a1a5-0c3255d31c50 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:45.825349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a915676-e7ea-4cd0-a1a5-0c3255d31c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:45.845701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-6a915676-e7ea-4cd0-a1a5-0c3255d31c50 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:45.847014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-6a915676-e7ea-4cd0-a1a5-0c3255d31c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:45.915284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1225/2419] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:39:45.924932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef67ad-3d5e-45e3-8388-9e2f6240145d admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:45.927108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-dfef67ad-3d5e-45e3-8388-9e2f6240145d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:45.993451 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef67ad-3d5e-45e3-8388-9e2f6240145d admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:45.997988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d03a77-950f-4679-9db6-c8a6716e7df6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:45.998601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d03a77-950f-4679-9db6-c8a6716e7df6 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:45.998908 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d03a77-950f-4679-9db6-c8a6716e7df6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:45.999201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d03a77-950f-4679-9db6-c8a6716e7df6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:46.009481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d03a77-950f-4679-9db6-c8a6716e7df6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:46.010948 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d03a77-950f-4679-9db6-c8a6716e7df6 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:46.023238 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1195/2420] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:39:47.578833 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc5a1c-d59e-4ae4-921f-5b2ee0ff755f None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:47.579418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc5a1c-d59e-4ae4-921f-5b2ee0ff755f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:47.579746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc5a1c-d59e-4ae4-921f-5b2ee0ff755f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:47.600496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:39:47.610075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-61cc5a1c-d59e-4ae4-921f-5b2ee0ff755f None None] MFA Rules not processed for user `dac33ec33c26411d9dcf8c3bbea7a9e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:47.611320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-61cc5a1c-d59e-4ae4-921f-5b2ee0ff755f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:47.680438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1226/2421] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:39:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:39:47.690835 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d03a77-950f-4679-9db6-c8a6716e7df6 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:47.692231 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-97d03a77-950f-4679-9db6-c8a6716e7df6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:47.773105 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d03a77-950f-4679-9db6-c8a6716e7df6 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:47.776732 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613750ba-0adf-4fd1-a9ce-213d741382e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:47.777448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613750ba-0adf-4fd1-a9ce-213d741382e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:47.777621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613750ba-0adf-4fd1-a9ce-213d741382e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:47.777892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613750ba-0adf-4fd1-a9ce-213d741382e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:47.787390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613750ba-0adf-4fd1-a9ce-213d741382e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:47.802602 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613750ba-0adf-4fd1-a9ce-213d741382e0 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:47.811696 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1196/2422] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:39:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:39:50.948016 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d2d85-2578-43e7-9032-614f431a5529 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:50.953498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d2d85-2578-43e7-9032-614f431a5529 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:50.953498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d2d85-2578-43e7-9032-614f431a5529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:50.968706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-0a3d2d85-2578-43e7-9032-614f431a5529 None None] MFA Rules not processed for user `dac33ec33c26411d9dcf8c3bbea7a9e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:39:50.970183 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0a3d2d85-2578-43e7-9032-614f431a5529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:50.977206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1227/2423] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:39:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:39:51.211465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613750ba-0adf-4fd1-a9ce-213d741382e0 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:51.221399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613750ba-0adf-4fd1-a9ce-213d741382e0 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:51.224912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136b840-a8f5-442b-b9bb-ccb83b320761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:51.225496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136b840-a8f5-442b-b9bb-ccb83b320761 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:51.225771 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136b840-a8f5-442b-b9bb-ccb83b320761 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:51.226039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136b840-a8f5-442b-b9bb-ccb83b320761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:51.227745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-5136b840-a8f5-442b-b9bb-ccb83b320761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:51.236743 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5136b840-a8f5-442b-b9bb-ccb83b320761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:51.238147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5136b840-a8f5-442b-b9bb-ccb83b320761 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:51.245804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1197/2424] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:39:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:39:57.260737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d2d85-2578-43e7-9032-614f431a5529 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:57.276070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d2d85-2578-43e7-9032-614f431a5529 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:57.281886 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d11e0b-b694-4fff-8cd9-2eddc41722ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:57.283127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d11e0b-b694-4fff-8cd9-2eddc41722ea service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:57.283600 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d11e0b-b694-4fff-8cd9-2eddc41722ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:57.284037 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d11e0b-b694-4fff-8cd9-2eddc41722ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:57.286881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b3d11e0b-b694-4fff-8cd9-2eddc41722ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:57.299358 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d11e0b-b694-4fff-8cd9-2eddc41722ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:57.300804 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d11e0b-b694-4fff-8cd9-2eddc41722ea service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:57.311230 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1228/2425] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:39:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:39:59.382302 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136b840-a8f5-442b-b9bb-ccb83b320761 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:39:59.383876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-5136b840-a8f5-442b-b9bb-ccb83b320761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:59.391719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:39:59.462259 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136b840-a8f5-442b-b9bb-ccb83b320761 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:39:59.467552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649fcb5-bf23-49a0-a8fa-c3907922326d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:39:59.468758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649fcb5-bf23-49a0-a8fa-c3907922326d service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:39:59.469039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649fcb5-bf23-49a0-a8fa-c3907922326d service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:39:59.469577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649fcb5-bf23-49a0-a8fa-c3907922326d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:39:59.471395 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-5649fcb5-bf23-49a0-a8fa-c3907922326d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:39:59.483218 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649fcb5-bf23-49a0-a8fa-c3907922326d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:39:59.484792 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649fcb5-bf23-49a0-a8fa-c3907922326d service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:39:59.495006 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1198/2426] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:39:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:40:01.810121 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d11e0b-b694-4fff-8cd9-2eddc41722ea service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:40:01.819682 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d11e0b-b694-4fff-8cd9-2eddc41722ea service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:40:01.823371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414213e-3c3b-43fd-b32f-58887e46ba23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:40:01.823963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414213e-3c3b-43fd-b32f-58887e46ba23 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:40:01.824243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414213e-3c3b-43fd-b32f-58887e46ba23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:40:01.824514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414213e-3c3b-43fd-b32f-58887e46ba23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:40:01.826222 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f414213e-3c3b-43fd-b32f-58887e46ba23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:40:01.887707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414213e-3c3b-43fd-b32f-58887e46ba23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:40:01.889132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414213e-3c3b-43fd-b32f-58887e46ba23 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:40:01.898994 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1229/2427] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:40:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:40:04.516026 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649fcb5-bf23-49a0-a8fa-c3907922326d service nova] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:40:04.527051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649fcb5-bf23-49a0-a8fa-c3907922326d service nova] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:40:04.532577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be64d49-581e-4536-a170-6e8da196bc26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:40:04.533186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be64d49-581e-4536-a170-6e8da196bc26 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:40:04.533493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be64d49-581e-4536-a170-6e8da196bc26 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:40:04.533773 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be64d49-581e-4536-a170-6e8da196bc26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:40:04.535557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-2be64d49-581e-4536-a170-6e8da196bc26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:40:04.548594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be64d49-581e-4536-a170-6e8da196bc26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:40:04.550181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be64d49-581e-4536-a170-6e8da196bc26 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:40:04.561157 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1199/2428] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:40:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:40:07.763725 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414213e-3c3b-43fd-b32f-58887e46ba23 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:40:07.765991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f414213e-3c3b-43fd-b32f-58887e46ba23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:40:07.782911 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414213e-3c3b-43fd-b32f-58887e46ba23 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:40:07.789373 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54502e7-d06b-4a56-b61b-d8217b707b32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:40:07.790175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54502e7-d06b-4a56-b61b-d8217b707b32 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:40:07.790614 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54502e7-d06b-4a56-b61b-d8217b707b32 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:40:07.791097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54502e7-d06b-4a56-b61b-d8217b707b32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:40:07.793473 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c54502e7-d06b-4a56-b61b-d8217b707b32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:40:07.808619 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54502e7-d06b-4a56-b61b-d8217b707b32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:40:07.810222 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54502e7-d06b-4a56-b61b-d8217b707b32 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:40:07.821404 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1230/2429] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:40:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:40:14.556416 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=38 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:40:16.183437 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274436b-c476-49fc-97f5-fe81872dcc89 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:40:16.184043 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274436b-c476-49fc-97f5-fe81872dcc89 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:40:16.184142 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274436b-c476-49fc-97f5-fe81872dcc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:40:16.203870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-c274436b-c476-49fc-97f5-fe81872dcc89 None None] MFA Rules not processed for user `9136f02e6c664f27b3f4fd35475f0eea`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:40:16.205048 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c274436b-c476-49fc-97f5-fe81872dcc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:40:16.271567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1200/2430] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:40:16 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:40:17.816528 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=73 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:40:18.046700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18ca17-6079-45a6-9842-5e45be884f4a None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:40:18.047059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18ca17-6079-45a6-9842-5e45be884f4a None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:40:18.047339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18ca17-6079-45a6-9842-5e45be884f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:40:18.068386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-9d18ca17-6079-45a6-9842-5e45be884f4a None None] MFA Rules not processed for user `86300433c724483994f59709a3f015db`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:40:18.069534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-9d18ca17-6079-45a6-9842-5e45be884f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:40:18.134937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1231/2431] 10.222.0.15 () {58 vars in 967 bytes} [Mon Sep 5 14:40:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:40:18.157004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274436b-c476-49fc-97f5-fe81872dcc89 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:40:18.158509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c274436b-c476-49fc-97f5-fe81872dcc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:40:18.229609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274436b-c476-49fc-97f5-fe81872dcc89 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:40:18.234656 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbd390-f15e-46bb-9b11-5e82c6edb752 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a85809d14cb449d848f22a3f39bf641', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:40:18.235298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbd390-f15e-46bb-9b11-5e82c6edb752 service swift] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:40:18.235581 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbd390-f15e-46bb-9b11-5e82c6edb752 service swift] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:40:18.235856 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbd390-f15e-46bb-9b11-5e82c6edb752 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:40:18.243900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbd390-f15e-46bb-9b11-5e82c6edb752 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:40:18.245484 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbd390-f15e-46bb-9b11-5e82c6edb752 service swift] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:40:18.253864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1201/2432] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 5 14:40:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 14:40:25.101469 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18ca17-6079-45a6-9842-5e45be884f4a None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:40:25.113978 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18ca17-6079-45a6-9842-5e45be884f4a None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:40:25.119339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7e989-dee8-400e-9d91-241255323526 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:40:25.119930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7e989-dee8-400e-9d91-241255323526 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:40:25.120206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7e989-dee8-400e-9d91-241255323526 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:40:25.120470 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7e989-dee8-400e-9d91-241255323526 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:40:25.122175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e2a7e989-dee8-400e-9d91-241255323526 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:40:25.180652 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7e989-dee8-400e-9d91-241255323526 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:40:25.182231 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7e989-dee8-400e-9d91-241255323526 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:40:25.191867 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1232/2433] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:40:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:40:28.250719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=57 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:40:35.187042 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=53 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:40:43.301321 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efc695-db5b-45da-a963-0b92bcddb111 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:40:43.301812 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efc695-db5b-45da-a963-0b92bcddb111 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:40:43.301924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efc695-db5b-45da-a963-0b92bcddb111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:40:43.320769 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-f6efc695-db5b-45da-a963-0b92bcddb111 None None] MFA Rules not processed for user `c078e655c9454316aa7afe13a6b565e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:40:43.321954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f6efc695-db5b-45da-a963-0b92bcddb111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:40:43.328872 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1202/2434] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:40:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:40:43.846415 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7e989-dee8-400e-9d91-241255323526 service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:40:43.855737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7e989-dee8-400e-9d91-241255323526 service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:40:43.859549 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57dce5-f316-4589-84d5-36039006f285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:40:43.860146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57dce5-f316-4589-84d5-36039006f285 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:40:43.860422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57dce5-f316-4589-84d5-36039006f285 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:40:43.860694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57dce5-f316-4589-84d5-36039006f285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:40:43.862466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ba57dce5-f316-4589-84d5-36039006f285 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:40:43.871765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57dce5-f316-4589-84d5-36039006f285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:40:43.873167 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57dce5-f316-4589-84d5-36039006f285 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:40:43.881594 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1233/2435] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:40:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:40:53.316025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:40:53.877991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=6 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:41:32.445999 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c548a63-3b65-495c-8017-52c1b9a134d4 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:41:32.446558 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c548a63-3b65-495c-8017-52c1b9a134d4 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:41:32.446643 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c548a63-3b65-495c-8017-52c1b9a134d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:41:32.466140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-9c548a63-3b65-495c-8017-52c1b9a134d4 None None] MFA Rules not processed for user `dac33ec33c26411d9dcf8c3bbea7a9e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:41:32.467529 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9c548a63-3b65-495c-8017-52c1b9a134d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:41:32.474624 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1203/2436] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:41:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:41:42.460419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:42:59.051132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57dce5-f316-4589-84d5-36039006f285 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:42:59.052595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ba57dce5-f316-4589-84d5-36039006f285 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:42:59.213166 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57dce5-f316-4589-84d5-36039006f285 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:42:59.218360 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105850a-bd9a-4465-a43f-10b5dee9db6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:42:59.219003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105850a-bd9a-4465-a43f-10b5dee9db6b service placement] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:42:59.219285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105850a-bd9a-4465-a43f-10b5dee9db6b service placement] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:42:59.219590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105850a-bd9a-4465-a43f-10b5dee9db6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:42:59.221329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0105850a-bd9a-4465-a43f-10b5dee9db6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:42:59.233644 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0105850a-bd9a-4465-a43f-10b5dee9db6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:42:59.235470 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0105850a-bd9a-4465-a43f-10b5dee9db6b service placement] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:42:59.245470 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1234/2437] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:42:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:43:09.241102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=63 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:43:19.922916 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c548a63-3b65-495c-8017-52c1b9a134d4 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:43:19.967940 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c548a63-3b65-495c-8017-52c1b9a134d4 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:43:19.973676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a08054-5431-4d22-80ea-575d21f96893 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:43:19.974918 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a08054-5431-4d22-80ea-575d21f96893 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:43:19.975216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a08054-5431-4d22-80ea-575d21f96893 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:43:19.975488 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a08054-5431-4d22-80ea-575d21f96893 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:43:19.977210 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d9a08054-5431-4d22-80ea-575d21f96893 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:43:20.033565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a08054-5431-4d22-80ea-575d21f96893 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:43:20.035206 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a08054-5431-4d22-80ea-575d21f96893 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:43:20.043793 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1204/2438] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:43:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:43:30.039807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=43 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:43:44.947238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105850a-bd9a-4465-a43f-10b5dee9db6b service placement] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:43:44.959326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105850a-bd9a-4465-a43f-10b5dee9db6b service placement] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:43:44.964683 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b194ccb-112b-4e75-83bf-90b4b90f7421 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:43:44.965277 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b194ccb-112b-4e75-83bf-90b4b90f7421 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:43:44.965559 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b194ccb-112b-4e75-83bf-90b4b90f7421 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:43:44.965836 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b194ccb-112b-4e75-83bf-90b4b90f7421 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:43:44.968049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-2b194ccb-112b-4e75-83bf-90b4b90f7421 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:43:45.024490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b194ccb-112b-4e75-83bf-90b4b90f7421 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:43:45.026054 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b194ccb-112b-4e75-83bf-90b4b90f7421 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:43:45.035288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1235/2439] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:43:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:43:55.031314 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=28 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:44:13.412776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a08054-5431-4d22-80ea-575d21f96893 service nova] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:44:13.431479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a08054-5431-4d22-80ea-575d21f96893 service nova] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:44:13.435051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019304b-d81e-4c0d-a7bd-14fef8fa0259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:44:13.435643 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019304b-d81e-4c0d-a7bd-14fef8fa0259 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:44:13.435916 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019304b-d81e-4c0d-a7bd-14fef8fa0259 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:44:13.436183 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019304b-d81e-4c0d-a7bd-14fef8fa0259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:44:13.438417 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a019304b-d81e-4c0d-a7bd-14fef8fa0259 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:44:13.463657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019304b-d81e-4c0d-a7bd-14fef8fa0259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:44:13.465074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019304b-d81e-4c0d-a7bd-14fef8fa0259 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:44:13.475812 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1205/2440] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:44:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:44:23.471016 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=15 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:44:25.674180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b194ccb-112b-4e75-83bf-90b4b90f7421 service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:44:25.685331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b194ccb-112b-4e75-83bf-90b4b90f7421 service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:44:25.690829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0601531-bf90-43e4-ae83-0ebd0a090de7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:44:25.691426 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0601531-bf90-43e4-ae83-0ebd0a090de7 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:44:25.691706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0601531-bf90-43e4-ae83-0ebd0a090de7 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:44:25.691980 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0601531-bf90-43e4-ae83-0ebd0a090de7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:44:25.693672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b0601531-bf90-43e4-ae83-0ebd0a090de7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:44:25.772014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0601531-bf90-43e4-ae83-0ebd0a090de7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:44:25.773587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0601531-bf90-43e4-ae83-0ebd0a090de7 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:44:25.783750 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1236/2441] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:44:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:44:26.357210 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019304b-d81e-4c0d-a7bd-14fef8fa0259 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:44:26.365454 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019304b-d81e-4c0d-a7bd-14fef8fa0259 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:44:26.368985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9aca6f-bea2-48ff-8930-b404032ab467 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:44:26.369572 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9aca6f-bea2-48ff-8930-b404032ab467 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:44:26.369846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9aca6f-bea2-48ff-8930-b404032ab467 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:44:26.370115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9aca6f-bea2-48ff-8930-b404032ab467 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:44:26.371814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9f9aca6f-bea2-48ff-8930-b404032ab467 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:44:26.385675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9aca6f-bea2-48ff-8930-b404032ab467 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:44:26.387208 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9aca6f-bea2-48ff-8930-b404032ab467 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:44:26.398528 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1206/2442] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:44:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:44:29.990056 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0601531-bf90-43e4-ae83-0ebd0a090de7 service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:44:30.001947 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0601531-bf90-43e4-ae83-0ebd0a090de7 service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:44:30.007547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575dc9d-f437-4c73-8e97-f0269bfcc4d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:44:30.008415 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575dc9d-f437-4c73-8e97-f0269bfcc4d0 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:44:30.008704 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575dc9d-f437-4c73-8e97-f0269bfcc4d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:44:30.008976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575dc9d-f437-4c73-8e97-f0269bfcc4d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:44:30.011172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7575dc9d-f437-4c73-8e97-f0269bfcc4d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:44:30.053774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575dc9d-f437-4c73-8e97-f0269bfcc4d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:44:30.055415 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575dc9d-f437-4c73-8e97-f0269bfcc4d0 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:44:30.075910 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1237/2443] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:44:29 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:44:36.393773 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=6 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:44:40.072929 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=56 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:44:57.087161 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9aca6f-bea2-48ff-8930-b404032ab467 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:44:57.088551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9f9aca6f-bea2-48ff-8930-b404032ab467 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:44:57.120583 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9aca6f-bea2-48ff-8930-b404032ab467 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:44:57.124109 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce493b4-f961-4cee-af7c-b644b81f5820 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:44:57.124700 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce493b4-f961-4cee-af7c-b644b81f5820 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:44:57.124974 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce493b4-f961-4cee-af7c-b644b81f5820 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:44:57.125242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce493b4-f961-4cee-af7c-b644b81f5820 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:44:57.134318 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce493b4-f961-4cee-af7c-b644b81f5820 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:44:57.135794 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce493b4-f961-4cee-af7c-b644b81f5820 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:44:57.145492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1207/2444] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:44:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:45:07.140557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=15 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:45:23.338454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575dc9d-f437-4c73-8e97-f0269bfcc4d0 service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:45:23.348118 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575dc9d-f437-4c73-8e97-f0269bfcc4d0 service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:45:23.352462 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ea851-550f-4613-986a-4128729fadc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:45:23.353083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ea851-550f-4613-986a-4128729fadc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:45:23.353366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ea851-550f-4613-986a-4128729fadc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:45:23.353643 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ea851-550f-4613-986a-4128729fadc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:45:23.360738 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ea851-550f-4613-986a-4128729fadc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:45:23.362160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ea851-550f-4613-986a-4128729fadc7 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:45:23.370621 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1238/2445] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:45:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:45:33.367267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=6 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:46:57.252426 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce493b4-f961-4cee-af7c-b644b81f5820 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:46:57.261025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce493b4-f961-4cee-af7c-b644b81f5820 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:46:57.265119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e3639-8f27-4149-8622-bb0c5fda0b40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:46:57.265705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e3639-8f27-4149-8622-bb0c5fda0b40 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:46:57.265987 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e3639-8f27-4149-8622-bb0c5fda0b40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:46:57.266259 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e3639-8f27-4149-8622-bb0c5fda0b40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:46:57.274399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e3639-8f27-4149-8622-bb0c5fda0b40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:46:57.275854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e3639-8f27-4149-8622-bb0c5fda0b40 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:46:57.302558 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1208/2446] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:46:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:47:07.300376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=13 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:48:20.063625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ea851-550f-4613-986a-4128729fadc7 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:48:20.076729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ea851-550f-4613-986a-4128729fadc7 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:48:20.082885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67235e73-7b96-4e01-b5c9-bf01041e836b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:48:20.083875 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67235e73-7b96-4e01-b5c9-bf01041e836b service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:48:20.083875 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67235e73-7b96-4e01-b5c9-bf01041e836b service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:48:20.084071 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67235e73-7b96-4e01-b5c9-bf01041e836b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:48:20.092475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67235e73-7b96-4e01-b5c9-bf01041e836b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:48:20.094064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67235e73-7b96-4e01-b5c9-bf01041e836b service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:48:20.123452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1239/2447] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:48:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:48:30.120826 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=13 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:48:34.682840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e3639-8f27-4149-8622-bb0c5fda0b40 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:48:34.694221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e3639-8f27-4149-8622-bb0c5fda0b40 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:48:34.699220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac494e-ba3c-4e93-98b4-3a44335f25c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:48:34.703626 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac494e-ba3c-4e93-98b4-3a44335f25c3 service placement] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:48:34.703626 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac494e-ba3c-4e93-98b4-3a44335f25c3 service placement] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:48:34.703626 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac494e-ba3c-4e93-98b4-3a44335f25c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:48:34.709183 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac494e-ba3c-4e93-98b4-3a44335f25c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:48:34.710753 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac494e-ba3c-4e93-98b4-3a44335f25c3 service placement] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:48:34.720315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1209/2448] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:48:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:48:44.716294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=6 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:48:44.751717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67235e73-7b96-4e01-b5c9-bf01041e836b service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:48:44.763479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67235e73-7b96-4e01-b5c9-bf01041e836b service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:48:44.768933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb026c2a-aab7-4993-bdf9-e691946a1593 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:48:44.770178 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb026c2a-aab7-4993-bdf9-e691946a1593 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:48:44.770460 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb026c2a-aab7-4993-bdf9-e691946a1593 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:48:44.770729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb026c2a-aab7-4993-bdf9-e691946a1593 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:48:44.778298 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb026c2a-aab7-4993-bdf9-e691946a1593 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:48:44.779882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb026c2a-aab7-4993-bdf9-e691946a1593 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:48:44.805694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1240/2449] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:48:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:48:54.803209 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=13 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:49:11.144689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac494e-ba3c-4e93-98b4-3a44335f25c3 service placement] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:49:11.157073 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac494e-ba3c-4e93-98b4-3a44335f25c3 service placement] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:49:11.162555 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c944934-74b5-4fe1-a2b0-cc91a7d8bd8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:49:11.163189 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c944934-74b5-4fe1-a2b0-cc91a7d8bd8b service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:49:11.163465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c944934-74b5-4fe1-a2b0-cc91a7d8bd8b service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:49:11.163733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c944934-74b5-4fe1-a2b0-cc91a7d8bd8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:49:11.173617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c944934-74b5-4fe1-a2b0-cc91a7d8bd8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:49:11.175221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c944934-74b5-4fe1-a2b0-cc91a7d8bd8b service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:49:11.186004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1210/2450] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:49:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:49:12.721464 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb026c2a-aab7-4993-bdf9-e691946a1593 service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:49:12.734938 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb026c2a-aab7-4993-bdf9-e691946a1593 service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:49:12.738971 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15732e4-d17f-4fe2-82e2-e8f165c786f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:49:12.739623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15732e4-d17f-4fe2-82e2-e8f165c786f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:49:12.739905 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15732e4-d17f-4fe2-82e2-e8f165c786f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:49:12.741077 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15732e4-d17f-4fe2-82e2-e8f165c786f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:49:12.752109 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15732e4-d17f-4fe2-82e2-e8f165c786f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:49:12.753857 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15732e4-d17f-4fe2-82e2-e8f165c786f9 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:49:12.769948 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1241/2451] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:49:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:49:15.722854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda94cc-2b30-479e-bff5-765e5ba994fd None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:49:15.723404 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda94cc-2b30-479e-bff5-765e5ba994fd None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:49:15.723404 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda94cc-2b30-479e-bff5-765e5ba994fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:49:15.762021 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-4fda94cc-2b30-479e-bff5-765e5ba994fd None None] MFA Rules not processed for user `c078e655c9454316aa7afe13a6b565e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 14:49:15.763257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-4fda94cc-2b30-479e-bff5-765e5ba994fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:49:15.770478 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1211/2452] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 14:49:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 14:49:16.069733 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15732e4-d17f-4fe2-82e2-e8f165c786f9 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:49:16.079206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15732e4-d17f-4fe2-82e2-e8f165c786f9 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:49:16.082921 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994625f3-124f-4233-bf4f-82a95cbfbc98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:49:16.083513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994625f3-124f-4233-bf4f-82a95cbfbc98 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:49:16.083790 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994625f3-124f-4233-bf4f-82a95cbfbc98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:49:16.084059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994625f3-124f-4233-bf4f-82a95cbfbc98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:49:16.091137 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994625f3-124f-4233-bf4f-82a95cbfbc98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:49:16.092547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994625f3-124f-4233-bf4f-82a95cbfbc98 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:49:16.100643 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1242/2453] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:49:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:49:17.585947 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda94cc-2b30-479e-bff5-765e5ba994fd None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:49:17.597544 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda94cc-2b30-479e-bff5-765e5ba994fd None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:49:17.602368 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff13885-8a26-4df2-9606-da03877a21c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:49:17.602985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff13885-8a26-4df2-9606-da03877a21c0 service placement] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:49:17.603269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff13885-8a26-4df2-9606-da03877a21c0 service placement] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:49:17.603541 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff13885-8a26-4df2-9606-da03877a21c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:49:17.605559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-2ff13885-8a26-4df2-9606-da03877a21c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:49:17.616842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff13885-8a26-4df2-9606-da03877a21c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:49:17.618414 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff13885-8a26-4df2-9606-da03877a21c0 service placement] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:49:17.628635 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1212/2454] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:49:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:49:26.097072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=12 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:49:27.623973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=23 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:49:31.558353 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994625f3-124f-4233-bf4f-82a95cbfbc98 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:49:31.572528 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994625f3-124f-4233-bf4f-82a95cbfbc98 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:49:31.578113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0f75d-bb75-4276-a074-e982d3095fd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:49:31.584710 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0f75d-bb75-4276-a074-e982d3095fd8 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:49:31.584710 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0f75d-bb75-4276-a074-e982d3095fd8 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:49:31.584710 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0f75d-bb75-4276-a074-e982d3095fd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:49:31.588248 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0f75d-bb75-4276-a074-e982d3095fd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:49:31.589886 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0f75d-bb75-4276-a074-e982d3095fd8 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:49:31.600347 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1243/2455] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:49:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:49:41.595955 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=6 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:49:57.705512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff13885-8a26-4df2-9606-da03877a21c0 service placement] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:49:57.717477 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff13885-8a26-4df2-9606-da03877a21c0 service placement] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:49:57.722752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946d3c4-7330-4b28-a5b1-32e4e90d0dc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:49:57.723364 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946d3c4-7330-4b28-a5b1-32e4e90d0dc7 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:49:57.723642 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946d3c4-7330-4b28-a5b1-32e4e90d0dc7 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:49:57.723911 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946d3c4-7330-4b28-a5b1-32e4e90d0dc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:49:57.725691 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-5946d3c4-7330-4b28-a5b1-32e4e90d0dc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:49:57.802278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946d3c4-7330-4b28-a5b1-32e4e90d0dc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:49:57.803881 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946d3c4-7330-4b28-a5b1-32e4e90d0dc7 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:49:57.821742 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1213/2456] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:49:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:50:07.819267 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=41 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:50:26.283813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0f75d-bb75-4276-a074-e982d3095fd8 service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:50:26.285746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-1db0f75d-bb75-4276-a074-e982d3095fd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:50:26.353745 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0f75d-bb75-4276-a074-e982d3095fd8 service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:50:26.357654 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2f54b-a144-4c50-8474-f403d8fe601d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:50:26.358284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2f54b-a144-4c50-8474-f403d8fe601d service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:50:26.358567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2f54b-a144-4c50-8474-f403d8fe601d service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:50:26.358873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2f54b-a144-4c50-8474-f403d8fe601d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:50:26.360687 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-29f2f54b-a144-4c50-8474-f403d8fe601d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:50:26.370485 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2f54b-a144-4c50-8474-f403d8fe601d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:50:26.379380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2f54b-a144-4c50-8474-f403d8fe601d service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:50:26.379858 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1244/2457] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:50:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:50:36.376342 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=31 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:50:57.749011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946d3c4-7330-4b28-a5b1-32e4e90d0dc7 service nova] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:50:57.802200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946d3c4-7330-4b28-a5b1-32e4e90d0dc7 service nova] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:50:57.805721 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fae4d9-e285-48b4-92a3-de77a1257c6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:50:57.806316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fae4d9-e285-48b4-92a3-de77a1257c6f service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:50:57.806594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fae4d9-e285-48b4-92a3-de77a1257c6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:50:57.806897 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fae4d9-e285-48b4-92a3-de77a1257c6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:50:57.872473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fae4d9-e285-48b4-92a3-de77a1257c6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:50:57.873907 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fae4d9-e285-48b4-92a3-de77a1257c6f service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:50:57.885333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1214/2458] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:50:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:50:58.000479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2f54b-a144-4c50-8474-f403d8fe601d service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:50:58.010562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2f54b-a144-4c50-8474-f403d8fe601d service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:50:58.014374 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876ff54-c775-4298-bb45-be3982e64ce8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:50:58.015006 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876ff54-c775-4298-bb45-be3982e64ce8 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:50:58.015291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876ff54-c775-4298-bb45-be3982e64ce8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:50:58.015564 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876ff54-c775-4298-bb45-be3982e64ce8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:50:58.024875 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876ff54-c775-4298-bb45-be3982e64ce8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:50:58.026309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876ff54-c775-4298-bb45-be3982e64ce8 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:50:58.044680 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1245/2459] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:50:57 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:50:58.096524 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fae4d9-e285-48b4-92a3-de77a1257c6f service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:50:58.105272 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fae4d9-e285-48b4-92a3-de77a1257c6f service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:50:58.109328 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a6942-6ce0-40c4-92ca-ca4cd2c22306 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:50:58.109933 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a6942-6ce0-40c4-92ca-ca4cd2c22306 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:50:58.110214 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a6942-6ce0-40c4-92ca-ca4cd2c22306 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:50:58.110487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a6942-6ce0-40c4-92ca-ca4cd2c22306 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:50:58.172922 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a6942-6ce0-40c4-92ca-ca4cd2c22306 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:50:58.174371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a6942-6ce0-40c4-92ca-ca4cd2c22306 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:50:58.185568 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1215/2460] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:50:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:51:08.042152 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=10 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:51:08.180420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=82 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:52:58.437949 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876ff54-c775-4298-bb45-be3982e64ce8 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:52:58.446389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876ff54-c775-4298-bb45-be3982e64ce8 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:52:58.450039 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb450ae6-2724-4b3a-80b5-1c2745f90314 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:52:58.450625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb450ae6-2724-4b3a-80b5-1c2745f90314 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:52:58.450998 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb450ae6-2724-4b3a-80b5-1c2745f90314 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:52:58.451271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb450ae6-2724-4b3a-80b5-1c2745f90314 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:52:58.453071 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-bb450ae6-2724-4b3a-80b5-1c2745f90314 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:52:58.514130 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb450ae6-2724-4b3a-80b5-1c2745f90314 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:52:58.515649 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb450ae6-2724-4b3a-80b5-1c2745f90314 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:52:58.524484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1246/2461] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:52:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:53:08.520356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=28 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:53:37.620612 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a6942-6ce0-40c4-92ca-ca4cd2c22306 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:53:37.622229 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-047a6942-6ce0-40c4-92ca-ca4cd2c22306 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:53:37.758200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a6942-6ce0-40c4-92ca-ca4cd2c22306 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:53:37.763521 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a6de6-c555-41fe-b2fe-49aacfdbed06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:53:37.764120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a6de6-c555-41fe-b2fe-49aacfdbed06 service placement] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:53:37.764398 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a6de6-c555-41fe-b2fe-49aacfdbed06 service placement] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:53:37.764668 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a6de6-c555-41fe-b2fe-49aacfdbed06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:53:37.766455 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-418a6de6-c555-41fe-b2fe-49aacfdbed06 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:53:37.777299 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a6de6-c555-41fe-b2fe-49aacfdbed06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:53:37.778901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a6de6-c555-41fe-b2fe-49aacfdbed06 service placement] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:53:37.789158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1216/2462] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:53:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:53:45.111657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb450ae6-2724-4b3a-80b5-1c2745f90314 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:53:45.113233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-bb450ae6-2724-4b3a-80b5-1c2745f90314 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:53:45.176717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb450ae6-2724-4b3a-80b5-1c2745f90314 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:53:45.182617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fecddb-0b24-40a1-b66c-00c74ddb5f14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:53:45.183244 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fecddb-0b24-40a1-b66c-00c74ddb5f14 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:53:45.183524 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fecddb-0b24-40a1-b66c-00c74ddb5f14 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:53:45.183822 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fecddb-0b24-40a1-b66c-00c74ddb5f14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:53:45.185613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-35fecddb-0b24-40a1-b66c-00c74ddb5f14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:53:45.253484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fecddb-0b24-40a1-b66c-00c74ddb5f14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:53:45.255635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fecddb-0b24-40a1-b66c-00c74ddb5f14 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:53:45.264330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1247/2463] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:53:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:53:47.785115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=63 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:53:55.260376 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=55 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:54:13.964410 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a6de6-c555-41fe-b2fe-49aacfdbed06 service placement] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:54:13.982412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a6de6-c555-41fe-b2fe-49aacfdbed06 service placement] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:54:13.986311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53d79f-78cc-4652-b8f3-89e73d1bcb1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:54:13.986940 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53d79f-78cc-4652-b8f3-89e73d1bcb1d service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:54:13.987221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53d79f-78cc-4652-b8f3-89e73d1bcb1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:54:13.987492 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53d79f-78cc-4652-b8f3-89e73d1bcb1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:54:13.989281 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-4e53d79f-78cc-4652-b8f3-89e73d1bcb1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:54:14.016128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53d79f-78cc-4652-b8f3-89e73d1bcb1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:54:14.017561 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53d79f-78cc-4652-b8f3-89e73d1bcb1d service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:54:14.028856 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1217/2464] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:54:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:54:24.023556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=15 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:54:59.340935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fecddb-0b24-40a1-b66c-00c74ddb5f14 service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:54:59.342503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-35fecddb-0b24-40a1-b66c-00c74ddb5f14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:54:59.375403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fecddb-0b24-40a1-b66c-00c74ddb5f14 service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:54:59.379489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4ca25-cd61-426c-8bc4-ff42eb7de6b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:54:59.380095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4ca25-cd61-426c-8bc4-ff42eb7de6b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:54:59.380371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4ca25-cd61-426c-8bc4-ff42eb7de6b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:54:59.380640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4ca25-cd61-426c-8bc4-ff42eb7de6b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:54:59.382477 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-60a4ca25-cd61-426c-8bc4-ff42eb7de6b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:54:59.455445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4ca25-cd61-426c-8bc4-ff42eb7de6b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:54:59.457534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4ca25-cd61-426c-8bc4-ff42eb7de6b1 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:54:59.466236 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1248/2465] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:54:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:54:59.591152 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53d79f-78cc-4652-b8f3-89e73d1bcb1d service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:54:59.600131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53d79f-78cc-4652-b8f3-89e73d1bcb1d service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:54:59.603645 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4db39b-85a8-4f98-bb8b-f8a7a2560e7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:54:59.604233 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4db39b-85a8-4f98-bb8b-f8a7a2560e7e service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:54:59.604510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4db39b-85a8-4f98-bb8b-f8a7a2560e7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:54:59.604779 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4db39b-85a8-4f98-bb8b-f8a7a2560e7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:54:59.613437 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4db39b-85a8-4f98-bb8b-f8a7a2560e7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:54:59.614878 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4db39b-85a8-4f98-bb8b-f8a7a2560e7e service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:54:59.623392 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1218/2466] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:54:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:55:02.474972 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4ca25-cd61-426c-8bc4-ff42eb7de6b1 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:55:02.476574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-60a4ca25-cd61-426c-8bc4-ff42eb7de6b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:55:02.492556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4ca25-cd61-426c-8bc4-ff42eb7de6b1 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:55:02.498299 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fcc94-6492-470d-b471-52a6b696d796 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:55:02.498943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fcc94-6492-470d-b471-52a6b696d796 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:55:02.499227 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fcc94-6492-470d-b471-52a6b696d796 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:55:02.499504 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fcc94-6492-470d-b471-52a6b696d796 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:55:02.501363 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e06fcc94-6492-470d-b471-52a6b696d796 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:55:02.514533 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06fcc94-6492-470d-b471-52a6b696d796 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:55:02.517484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06fcc94-6492-470d-b471-52a6b696d796 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:55:02.527511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1249/2467] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:55:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:55:09.619635 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=6 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:55:12.522989 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=48 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:55:13.188189 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4db39b-85a8-4f98-bb8b-f8a7a2560e7e service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:55:13.189670 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-7a4db39b-85a8-4f98-bb8b-f8a7a2560e7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:55:13.202564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4db39b-85a8-4f98-bb8b-f8a7a2560e7e service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:55:13.206207 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008359b-dab7-4a53-8fe7-b2ac038f412a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:55:13.206945 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008359b-dab7-4a53-8fe7-b2ac038f412a service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:55:13.207116 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008359b-dab7-4a53-8fe7-b2ac038f412a service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:55:13.207397 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008359b-dab7-4a53-8fe7-b2ac038f412a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:55:13.209152 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-2008359b-dab7-4a53-8fe7-b2ac038f412a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:55:13.295016 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008359b-dab7-4a53-8fe7-b2ac038f412a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:55:13.296451 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008359b-dab7-4a53-8fe7-b2ac038f412a service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:55:13.315385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1219/2468] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:55:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:55:16.750365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fcc94-6492-470d-b471-52a6b696d796 service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:55:16.763304 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fcc94-6492-470d-b471-52a6b696d796 service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:55:16.768372 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04e466-a4b3-4f0c-bac9-64db29d48478 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:55:16.770835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04e466-a4b3-4f0c-bac9-64db29d48478 service placement] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:55:16.770835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04e466-a4b3-4f0c-bac9-64db29d48478 service placement] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:55:16.770835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04e466-a4b3-4f0c-bac9-64db29d48478 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:55:16.772030 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-4c04e466-a4b3-4f0c-bac9-64db29d48478 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:55:16.783788 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04e466-a4b3-4f0c-bac9-64db29d48478 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:55:16.785386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04e466-a4b3-4f0c-bac9-64db29d48478 service placement] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:55:16.796170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1250/2469] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:55:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:55:20.502333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008359b-dab7-4a53-8fe7-b2ac038f412a service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:55:20.504458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-2008359b-dab7-4a53-8fe7-b2ac038f412a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:55:20.518320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008359b-dab7-4a53-8fe7-b2ac038f412a service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:55:20.526509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f43a22-5ece-4e10-a7c2-4f60f67ee646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:55:20.527553 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f43a22-5ece-4e10-a7c2-4f60f67ee646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:55:20.528059 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f43a22-5ece-4e10-a7c2-4f60f67ee646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:55:20.528585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f43a22-5ece-4e10-a7c2-4f60f67ee646 admin admin] PATH_INFO: `/v3/users/60b48954cad94ade83e974fa28063b7d` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:55:20.530332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f43a22-5ece-4e10-a7c2-4f60f67ee646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60b48954cad94ade83e974fa28063b7d)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:55:20.533079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f43a22-5ece-4e10-a7c2-4f60f67ee646 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:55:20.699397 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-62f43a22-5ece-4e10-a7c2-4f60f67ee646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60b48954cad94ade83e974fa28063b7d'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:20.782522 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-62f43a22-5ece-4e10-a7c2-4f60f67ee646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60b48954cad94ade83e974fa28063b7d'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:20.783225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-62f43a22-5ece-4e10-a7c2-4f60f67ee646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60b48954cad94ade83e974fa28063b7d'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:20.795473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-62f43a22-5ece-4e10-a7c2-4f60f67ee646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60b48954cad94ade83e974fa28063b7d'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:20.802623 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-62f43a22-5ece-4e10-a7c2-4f60f67ee646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60b48954cad94ade83e974fa28063b7d'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:20.808479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1220/2470] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:55:20 2022] DELETE /identity/v3/users/60b48954cad94ade83e974fa28063b7d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:55:21.042862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04e466-a4b3-4f0c-bac9-64db29d48478 service placement] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:55:21.063866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04e466-a4b3-4f0c-bac9-64db29d48478 service placement] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:55:21.126589 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:55:21.127447 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:55:21.127727 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:55:21.128006 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af admin admin] PATH_INFO: `/v3/projects/172738e7ad624a429835832020a92723` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:55:21.129173 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af admin admin] RBAC: Authorizing `identity:delete_project(project_id=172738e7ad624a429835832020a92723)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:55:21.131141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:55:21.251887 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '172738e7ad624a429835832020a92723', 'request_id': 'req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:21.257963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '172738e7ad624a429835832020a92723', 'request_id': 'req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:21.336538 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '172738e7ad624a429835832020a92723', 'request_id': 'req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:21.347095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af admin admin] The token cache is being invalidate because project 172738e7ad624a429835832020a92723 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:55:21.347384 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:21.348515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:21.350000 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1251/2471] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:55:21 2022] DELETE /identity/v3/projects/172738e7ad624a429835832020a92723 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:55:21.354116 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f43a22-5ece-4e10-a7c2-4f60f67ee646 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:55:21.357656 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-62f43a22-5ece-4e10-a7c2-4f60f67ee646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:55:21.369232 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f43a22-5ece-4e10-a7c2-4f60f67ee646 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:55:21.401579 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:55:21.424202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e94159-ebe7-4e77-be67-d0a212549566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:55:21.424830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e94159-ebe7-4e77-be67-d0a212549566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:55:21.425118 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e94159-ebe7-4e77-be67-d0a212549566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:55:21.425390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e94159-ebe7-4e77-be67-d0a212549566 admin admin] PATH_INFO: `/v3/users/d96837e658544bc7a698718c9feeef38` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:55:21.438038 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e94159-ebe7-4e77-be67-d0a212549566 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d96837e658544bc7a698718c9feeef38)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:55:21.440040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e94159-ebe7-4e77-be67-d0a212549566 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:55:21.613924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f7e94159-ebe7-4e77-be67-d0a212549566 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd96837e658544bc7a698718c9feeef38'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:21.674270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f7e94159-ebe7-4e77-be67-d0a212549566 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd96837e658544bc7a698718c9feeef38'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:21.674932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f7e94159-ebe7-4e77-be67-d0a212549566 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd96837e658544bc7a698718c9feeef38'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:21.687791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f7e94159-ebe7-4e77-be67-d0a212549566 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd96837e658544bc7a698718c9feeef38'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:21.694636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f7e94159-ebe7-4e77-be67-d0a212549566 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd96837e658544bc7a698718c9feeef38'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:21.703099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1221/2472] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 14:55:21 2022] DELETE /identity/v3/users/d96837e658544bc7a698718c9feeef38 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:55:21.919296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:55:21.920751 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:55:21.936175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26250db9-fc2f-4b7c-87d2-6ac13c75c1af admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:55:21.992061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1420f-fda1-4ac7-9f2c-7f361821a004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:55:21.992688 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1420f-fda1-4ac7-9f2c-7f361821a004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:55:21.992969 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1420f-fda1-4ac7-9f2c-7f361821a004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:55:21.993243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1420f-fda1-4ac7-9f2c-7f361821a004 admin admin] PATH_INFO: `/v3/projects/ac15c000034b45ebb9273d817021536d` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:55:22.003290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd1420f-fda1-4ac7-9f2c-7f361821a004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac15c000034b45ebb9273d817021536d)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:55:22.005724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd1420f-fda1-4ac7-9f2c-7f361821a004 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:55:22.016303 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:55:22.098103 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-dfd1420f-fda1-4ac7-9f2c-7f361821a004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac15c000034b45ebb9273d817021536d', 'request_id': 'req-dfd1420f-fda1-4ac7-9f2c-7f361821a004'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:22.106320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-dfd1420f-fda1-4ac7-9f2c-7f361821a004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac15c000034b45ebb9273d817021536d', 'request_id': 'req-dfd1420f-fda1-4ac7-9f2c-7f361821a004'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:22.165257 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-dfd1420f-fda1-4ac7-9f2c-7f361821a004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac15c000034b45ebb9273d817021536d', 'request_id': 'req-dfd1420f-fda1-4ac7-9f2c-7f361821a004'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:22.176779 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-dfd1420f-fda1-4ac7-9f2c-7f361821a004 admin admin] The token cache is being invalidate because project ac15c000034b45ebb9273d817021536d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 14:55:22.179937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-dfd1420f-fda1-4ac7-9f2c-7f361821a004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:22.179937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-dfd1420f-fda1-4ac7-9f2c-7f361821a004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 14:55:22.179937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1252/2473] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 14:55:21 2022] DELETE /identity/v3/projects/ac15c000034b45ebb9273d817021536d => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 14:55:30.325637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e94159-ebe7-4e77-be67-d0a212549566 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:55:30.327213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f7e94159-ebe7-4e77-be67-d0a212549566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:55:30.383864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e94159-ebe7-4e77-be67-d0a212549566 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:55:30.433428 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ba4fd-aa80-402b-a439-a5b680c7e3d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:55:30.434010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ba4fd-aa80-402b-a439-a5b680c7e3d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:55:30.434288 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ba4fd-aa80-402b-a439-a5b680c7e3d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:55:30.434556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ba4fd-aa80-402b-a439-a5b680c7e3d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:55:30.436323 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-008ba4fd-aa80-402b-a439-a5b680c7e3d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:55:30.446257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ba4fd-aa80-402b-a439-a5b680c7e3d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:55:30.447723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ba4fd-aa80-402b-a439-a5b680c7e3d1 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:55:30.456095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1222/2474] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:55:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:55:32.173144 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:55:40.452294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:55:58.163512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1420f-fda1-4ac7-9f2c-7f361821a004 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:55:58.165012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-dfd1420f-fda1-4ac7-9f2c-7f361821a004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:55:58.189371 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ba4fd-aa80-402b-a439-a5b680c7e3d1 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:55:58.190805 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-008ba4fd-aa80-402b-a439-a5b680c7e3d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:55:58.239915 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1420f-fda1-4ac7-9f2c-7f361821a004 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:55:58.245484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc0b98-99bc-4ed6-a4c5-0e9149f9671f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:55:58.246090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc0b98-99bc-4ed6-a4c5-0e9149f9671f service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:55:58.246372 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc0b98-99bc-4ed6-a4c5-0e9149f9671f service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:55:58.246650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc0b98-99bc-4ed6-a4c5-0e9149f9671f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:55:58.248485 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a9bc0b98-99bc-4ed6-a4c5-0e9149f9671f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:55:58.262768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ba4fd-aa80-402b-a439-a5b680c7e3d1 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:55:58.268151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82393c4a-4be2-43d3-8f0d-7c7828d878a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:55:58.268748 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82393c4a-4be2-43d3-8f0d-7c7828d878a5 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:55:58.269029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82393c4a-4be2-43d3-8f0d-7c7828d878a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:55:58.269345 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82393c4a-4be2-43d3-8f0d-7c7828d878a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:55:58.271076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-82393c4a-4be2-43d3-8f0d-7c7828d878a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:55:58.332001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc0b98-99bc-4ed6-a4c5-0e9149f9671f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:55:58.333894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc0b98-99bc-4ed6-a4c5-0e9149f9671f service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:55:58.352074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82393c4a-4be2-43d3-8f0d-7c7828d878a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:55:58.352777 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1253/2475] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:55:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:55:58.353628 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82393c4a-4be2-43d3-8f0d-7c7828d878a5 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:55:58.363784 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1223/2476] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:55:58 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 14:55:58.415854 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc0b98-99bc-4ed6-a4c5-0e9149f9671f service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:55:58.417442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a9bc0b98-99bc-4ed6-a4c5-0e9149f9671f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:55:58.429664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc0b98-99bc-4ed6-a4c5-0e9149f9671f service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:55:58.433170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f95ee-15bc-445c-b882-c22311a7cbfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:55:58.433773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f95ee-15bc-445c-b882-c22311a7cbfd service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:55:58.434051 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f95ee-15bc-445c-b882-c22311a7cbfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:55:58.434323 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f95ee-15bc-445c-b882-c22311a7cbfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:55:58.436122 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-3d6f95ee-15bc-445c-b882-c22311a7cbfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:55:58.451840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f95ee-15bc-445c-b882-c22311a7cbfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:55:58.453363 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f95ee-15bc-445c-b882-c22311a7cbfd service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:55:58.474527 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1254/2477] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:55:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:56:08.358572 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=65 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:56:08.460057 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=75 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:58:40.624779 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82393c4a-4be2-43d3-8f0d-7c7828d878a5 service nova] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:58:40.626280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-82393c4a-4be2-43d3-8f0d-7c7828d878a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:58:40.692739 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82393c4a-4be2-43d3-8f0d-7c7828d878a5 service nova] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:58:40.698443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c896828-a179-413d-93ea-9216c1543db8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:58:40.699081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c896828-a179-413d-93ea-9216c1543db8 service placement] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:58:40.699367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c896828-a179-413d-93ea-9216c1543db8 service placement] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:58:40.699644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c896828-a179-413d-93ea-9216c1543db8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:58:40.701389 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-7c896828-a179-413d-93ea-9216c1543db8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:58:40.712635 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c896828-a179-413d-93ea-9216c1543db8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:58:40.714200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c896828-a179-413d-93ea-9216c1543db8 service placement] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:58:40.723867 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1224/2478] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:58:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:58:50.719757 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=32 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:59:13.323799 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f95ee-15bc-445c-b882-c22311a7cbfd service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:59:13.333768 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f95ee-15bc-445c-b882-c22311a7cbfd service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:59:13.337362 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae8716-7d9f-47d6-8b57-881d92d46589 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:59:13.337960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae8716-7d9f-47d6-8b57-881d92d46589 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:59:13.338234 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae8716-7d9f-47d6-8b57-881d92d46589 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:59:13.338499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae8716-7d9f-47d6-8b57-881d92d46589 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:59:13.340228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-3eae8716-7d9f-47d6-8b57-881d92d46589 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:59:13.354232 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae8716-7d9f-47d6-8b57-881d92d46589 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:59:13.355689 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae8716-7d9f-47d6-8b57-881d92d46589 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:59:13.367652 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1255/2479] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:59:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:59:13.796237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c896828-a179-413d-93ea-9216c1543db8 service placement] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:59:13.797630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-7c896828-a179-413d-93ea-9216c1543db8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:59:13.851847 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c896828-a179-413d-93ea-9216c1543db8 service placement] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:59:13.855382 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3b872-d299-4df7-a1bc-bbf8ad0707bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:59:13.855976 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3b872-d299-4df7-a1bc-bbf8ad0707bc service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:59:13.856258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3b872-d299-4df7-a1bc-bbf8ad0707bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:59:13.857181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3b872-d299-4df7-a1bc-bbf8ad0707bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:59:13.858913 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-75c3b872-d299-4df7-a1bc-bbf8ad0707bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:59:13.923407 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3b872-d299-4df7-a1bc-bbf8ad0707bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 14:59:13.924831 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3b872-d299-4df7-a1bc-bbf8ad0707bc service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 14:59:13.942307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1225/2480] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:59:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 14:59:14.099200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae8716-7d9f-47d6-8b57-881d92d46589 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:59:14.108528 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae8716-7d9f-47d6-8b57-881d92d46589 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:59:14.119583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab8ae7-9e48-43a3-8db7-8587efb95a2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:59:14.119583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab8ae7-9e48-43a3-8db7-8587efb95a2f service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:59:14.119583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab8ae7-9e48-43a3-8db7-8587efb95a2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:59:14.119583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab8ae7-9e48-43a3-8db7-8587efb95a2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:59:14.119941 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5dab8ae7-9e48-43a3-8db7-8587efb95a2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 14:59:14.148556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application [None req-5dab8ae7-9e48-43a3-8db7-8587efb95a2f service cinder] Could not find project: 172738e7ad624a429835832020a92723.: keystone.exception.ProjectNotFound: Could not find project: 172738e7ad624a429835832020a92723. Sep 05 14:59:14.148556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 14:59:14.148556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 14:59:14.148556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 14:59:14.148556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 14:59:14.148556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 14:59:14.148556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 14:59:14.148556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 14:59:14.148556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 14:59:14.148556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 14:59:14.148556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 14:59:14.148556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 14:59:14.148556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 05 14:59:14.148556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 05 14:59:14.148556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 05 14:59:14.152308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 05 14:59:14.152308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 05 14:59:14.152308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 05 14:59:14.152308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:59:14.152308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:59:14.152308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 05 14:59:14.152308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 05 14:59:14.152308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:59:14.152308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:59:14.152308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:59:14.152308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:59:14.152308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:59:14.152308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:59:14.152308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:59:14.152308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:59:14.152308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:59:14.152308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:59:14.154554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:59:14.154554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:59:14.154554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:59:14.154554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:59:14.154554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 05 14:59:14.154554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 05 14:59:14.154554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 05 14:59:14.154554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 05 14:59:14.154554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 05 14:59:14.154554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 05 14:59:14.154554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 05 14:59:14.154554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 05 14:59:14.154554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 14:59:14.154554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 14:59:14.154554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 14:59:14.154554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 14:59:14.154554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 14:59:14.156088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 14:59:14.156088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 14:59:14.156088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application with Lock( Sep 05 14:59:14.156088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 14:59:14.156088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._enter() Sep 05 14:59:14.156088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 14:59:14.156088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 14:59:14.156088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 14:59:14.156088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.creator() Sep 05 14:59:14.156088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 14:59:14.156088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application created_value = creator( Sep 05 14:59:14.156088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 14:59:14.156088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 14:59:14.156088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 14:59:14.156088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 14:59:14.156088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 14:59:14.156088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 14:59:14.157570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 172738e7ad624a429835832020a92723. Sep 05 14:59:14.157570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: ERROR keystone.server.flask.application Sep 05 14:59:14.157570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1256/2481] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:59:14 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 05 14:59:23.939906 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=56 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:59:24.119874 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=10 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 14:59:59.977182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3b872-d299-4df7-a1bc-bbf8ad0707bc service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 14:59:59.989441 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3b872-d299-4df7-a1bc-bbf8ad0707bc service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 14:59:59.995213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ecc5fb-5c4f-4fbe-9ca9-f6830c790872 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 14:59:59.995808 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ecc5fb-5c4f-4fbe-9ca9-f6830c790872 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 14:59:59.996095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ecc5fb-5c4f-4fbe-9ca9-f6830c790872 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 14:59:59.996428 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ecc5fb-5c4f-4fbe-9ca9-f6830c790872 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 14:59:59.998118 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b1ecc5fb-5c4f-4fbe-9ca9-f6830c790872 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:00:00.002585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab8ae7-9e48-43a3-8db7-8587efb95a2f service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:00:00.016236 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab8ae7-9e48-43a3-8db7-8587efb95a2f service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:00:00.021743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef6999-d5e4-4961-9c59-980de92323a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:00:00.022335 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef6999-d5e4-4961-9c59-980de92323a7 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:00:00.022616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef6999-d5e4-4961-9c59-980de92323a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:00:00.022932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef6999-d5e4-4961-9c59-980de92323a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:00:00.024742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a1ef6999-d5e4-4961-9c59-980de92323a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:00:00.066677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ecc5fb-5c4f-4fbe-9ca9-f6830c790872 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:00:00.068296 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ecc5fb-5c4f-4fbe-9ca9-f6830c790872 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:00:00.088933 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1226/2482] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 14:59:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:00:00.112403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef6999-d5e4-4961-9c59-980de92323a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:00:00.114004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef6999-d5e4-4961-9c59-980de92323a7 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:00:00.132743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1257/2483] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:00:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:00:10.085975 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=36 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:00:10.130132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=41 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:00:36.319665 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ecc5fb-5c4f-4fbe-9ca9-f6830c790872 service nova] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:00:36.329160 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ecc5fb-5c4f-4fbe-9ca9-f6830c790872 service nova] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:00:36.332731 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75757386-db40-43f8-8d35-65761e311bfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:00:36.333330 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75757386-db40-43f8-8d35-65761e311bfa service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:00:36.333607 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75757386-db40-43f8-8d35-65761e311bfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:00:36.333876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75757386-db40-43f8-8d35-65761e311bfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:00:36.341868 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75757386-db40-43f8-8d35-65761e311bfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:00:36.343420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75757386-db40-43f8-8d35-65761e311bfa service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:00:36.351519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1227/2484] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:00:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:00:46.347886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=6 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:00:57.728850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef6999-d5e4-4961-9c59-980de92323a7 service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:00:57.739221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef6999-d5e4-4961-9c59-980de92323a7 service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:00:57.743279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794510f-18ce-4e26-8069-643fd0a0984a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:00:57.743894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794510f-18ce-4e26-8069-643fd0a0984a service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:00:57.744171 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794510f-18ce-4e26-8069-643fd0a0984a service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:00:57.744484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794510f-18ce-4e26-8069-643fd0a0984a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:00:57.755739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794510f-18ce-4e26-8069-643fd0a0984a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:00:57.757163 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794510f-18ce-4e26-8069-643fd0a0984a service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:00:57.768101 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1258/2485] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:00:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:00:57.887650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75757386-db40-43f8-8d35-65761e311bfa service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:00:57.900333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75757386-db40-43f8-8d35-65761e311bfa service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:00:57.906215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd11b50-ed7d-4e5c-8303-f9d8c91ef79e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:00:57.906833 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd11b50-ed7d-4e5c-8303-f9d8c91ef79e service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:00:57.907112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd11b50-ed7d-4e5c-8303-f9d8c91ef79e service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:00:57.907383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd11b50-ed7d-4e5c-8303-f9d8c91ef79e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:00:57.915857 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd11b50-ed7d-4e5c-8303-f9d8c91ef79e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:00:57.917432 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd11b50-ed7d-4e5c-8303-f9d8c91ef79e service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:00:57.922450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794510f-18ce-4e26-8069-643fd0a0984a service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:00:57.927576 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1228/2486] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:00:57 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:00:57.935420 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794510f-18ce-4e26-8069-643fd0a0984a service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:00:57.940888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7accf-c18c-4d67-9c18-d2f6bfbdc5b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:00:57.941659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7accf-c18c-4d67-9c18-d2f6bfbdc5b2 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:00:57.941939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7accf-c18c-4d67-9c18-d2f6bfbdc5b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:00:57.942216 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7accf-c18c-4d67-9c18-d2f6bfbdc5b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:00:57.950233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea7accf-c18c-4d67-9c18-d2f6bfbdc5b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:00:57.951867 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea7accf-c18c-4d67-9c18-d2f6bfbdc5b2 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:00:57.960917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1259/2487] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:00:57 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 15:01:01.699034 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd11b50-ed7d-4e5c-8303-f9d8c91ef79e service nova] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:01:01.711590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd11b50-ed7d-4e5c-8303-f9d8c91ef79e service nova] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:01:01.718063 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d14037-4be8-4836-a424-a3a52adea7e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:01:01.718744 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d14037-4be8-4836-a424-a3a52adea7e4 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:01:01.719132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d14037-4be8-4836-a424-a3a52adea7e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:01:01.719511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d14037-4be8-4836-a424-a3a52adea7e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:01:01.721377 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-71d14037-4be8-4836-a424-a3a52adea7e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:01:01.734464 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d14037-4be8-4836-a424-a3a52adea7e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:01:01.736087 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d14037-4be8-4836-a424-a3a52adea7e4 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:01:01.747618 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1229/2488] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:01:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:01:04.275795 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5aef4-3550-493d-8b72-e35a0d535cf8 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:01:04.276301 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5aef4-3550-493d-8b72-e35a0d535cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:01:04.279565 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5aef4-3550-493d-8b72-e35a0d535cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:01:04.304271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-dfb5aef4-3550-493d-8b72-e35a0d535cf8 None None] MFA Rules not processed for user `91c691b6a7724d708bfc7424908516c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:01:04.305661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-dfb5aef4-3550-493d-8b72-e35a0d535cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:01:04.323302 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1260/2489] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:01:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 15:01:04.624044 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d14037-4be8-4836-a424-a3a52adea7e4 service nova] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:01:04.633265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d14037-4be8-4836-a424-a3a52adea7e4 service nova] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:01:04.636829 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdcea3-fc64-4d36-ad48-4050460a8a40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:01:04.637419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdcea3-fc64-4d36-ad48-4050460a8a40 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:01:04.637700 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdcea3-fc64-4d36-ad48-4050460a8a40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:01:04.646374 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdcea3-fc64-4d36-ad48-4050460a8a40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:01:04.646374 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b8fdcea3-fc64-4d36-ad48-4050460a8a40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:01:04.662805 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fdcea3-fc64-4d36-ad48-4050460a8a40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:01:04.664258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fdcea3-fc64-4d36-ad48-4050460a8a40 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:01:04.672276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1230/2490] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:01:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:01:06.088768 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5aef4-3550-493d-8b72-e35a0d535cf8 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:01:06.090154 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-dfb5aef4-3550-493d-8b72-e35a0d535cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:01:06.105645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5aef4-3550-493d-8b72-e35a0d535cf8 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:01:06.111249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8355371-29f7-43f3-8c5b-894e9e050cb7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:01:06.111885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8355371-29f7-43f3-8c5b-894e9e050cb7 service placement] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:01:06.112168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8355371-29f7-43f3-8c5b-894e9e050cb7 service placement] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:01:06.112443 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8355371-29f7-43f3-8c5b-894e9e050cb7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:01:06.114186 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a8355371-29f7-43f3-8c5b-894e9e050cb7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:01:06.125561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8355371-29f7-43f3-8c5b-894e9e050cb7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:01:06.127184 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8355371-29f7-43f3-8c5b-894e9e050cb7 service placement] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:01:06.136875 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1261/2491] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:01:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:01:13.208748 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdcea3-fc64-4d36-ad48-4050460a8a40 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:01:13.223154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdcea3-fc64-4d36-ad48-4050460a8a40 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:01:13.228952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bf6b1-7bd4-46a5-9b07-d9dfc3479307 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:01:13.229565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bf6b1-7bd4-46a5-9b07-d9dfc3479307 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:01:13.229855 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bf6b1-7bd4-46a5-9b07-d9dfc3479307 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:01:13.230174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bf6b1-7bd4-46a5-9b07-d9dfc3479307 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:01:13.232474 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-326bf6b1-7bd4-46a5-9b07-d9dfc3479307 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:01:13.245355 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326bf6b1-7bd4-46a5-9b07-d9dfc3479307 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:01:13.247023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326bf6b1-7bd4-46a5-9b07-d9dfc3479307 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:01:13.259223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1231/2492] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:01:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:01:16.132798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=28 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:01:18.728132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8355371-29f7-43f3-8c5b-894e9e050cb7 service placement] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:01:18.740214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8355371-29f7-43f3-8c5b-894e9e050cb7 service placement] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:01:18.745958 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1004b2-2235-446d-a90e-0cffea8a0c56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:01:18.746666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1004b2-2235-446d-a90e-0cffea8a0c56 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:01:18.747511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1004b2-2235-446d-a90e-0cffea8a0c56 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:01:18.747791 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1004b2-2235-446d-a90e-0cffea8a0c56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:01:18.749485 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-2b1004b2-2235-446d-a90e-0cffea8a0c56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:01:18.761478 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1004b2-2235-446d-a90e-0cffea8a0c56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:01:18.763735 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1004b2-2235-446d-a90e-0cffea8a0c56 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:01:18.774109 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1262/2493] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:01:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:01:23.254034 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=29 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:01:28.769673 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=6 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:03:14.130486 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c41fed-d2c5-46f0-89e9-6a20a2c70dd3 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:03:14.131049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c41fed-d2c5-46f0-89e9-6a20a2c70dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:03:14.131169 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c41fed-d2c5-46f0-89e9-6a20a2c70dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:03:14.190358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-32c41fed-d2c5-46f0-89e9-6a20a2c70dd3 None None] MFA Rules not processed for user `86300433c724483994f59709a3f015db`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:03:14.191608 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-32c41fed-d2c5-46f0-89e9-6a20a2c70dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:03:14.253792 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1232/2494] 10.222.0.15 () {58 vars in 967 bytes} [Mon Sep 5 15:03:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:03:14.366762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1004b2-2235-446d-a90e-0cffea8a0c56 service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:03:14.368222 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-2b1004b2-2235-446d-a90e-0cffea8a0c56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:03:14.455741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1004b2-2235-446d-a90e-0cffea8a0c56 service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:03:14.460837 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f27f1-ad7a-4cf7-8c39-d52cf20b4235 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a85809d14cb449d848f22a3f39bf641', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:03:14.461923 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f27f1-ad7a-4cf7-8c39-d52cf20b4235 service swift] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:03:14.462206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f27f1-ad7a-4cf7-8c39-d52cf20b4235 service swift] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:03:14.462479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f27f1-ad7a-4cf7-8c39-d52cf20b4235 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:03:14.470381 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f27f1-ad7a-4cf7-8c39-d52cf20b4235 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:03:14.472005 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f27f1-ad7a-4cf7-8c39-d52cf20b4235 service swift] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:03:14.480538 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1263/2495] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 5 15:03:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 15:03:15.527992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c41fed-d2c5-46f0-89e9-6a20a2c70dd3 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:03:15.537374 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c41fed-d2c5-46f0-89e9-6a20a2c70dd3 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:03:15.540952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d36ed-ac13-4e12-ae4d-7a0b10a3a212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:03:15.541563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d36ed-ac13-4e12-ae4d-7a0b10a3a212 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:03:15.541841 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d36ed-ac13-4e12-ae4d-7a0b10a3a212 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:03:15.542108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d36ed-ac13-4e12-ae4d-7a0b10a3a212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:03:15.543877 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9b8d36ed-ac13-4e12-ae4d-7a0b10a3a212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:03:15.606240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d36ed-ac13-4e12-ae4d-7a0b10a3a212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:03:15.607742 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d36ed-ac13-4e12-ae4d-7a0b10a3a212 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:03:15.617193 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1233/2496] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:03:24.476701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=41 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:03:25.612968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=74 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:03:48.590995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f27f1-ad7a-4cf7-8c39-d52cf20b4235 service swift] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:03:48.649281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f27f1-ad7a-4cf7-8c39-d52cf20b4235 service swift] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:03:48.655033 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d04603-4f12-4419-982b-ab20ded7b1d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:03:48.655636 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d04603-4f12-4419-982b-ab20ded7b1d5 service placement] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:03:48.655922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d04603-4f12-4419-982b-ab20ded7b1d5 service placement] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:03:48.656199 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d04603-4f12-4419-982b-ab20ded7b1d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:03:48.664550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d04603-4f12-4419-982b-ab20ded7b1d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:03:48.666160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d04603-4f12-4419-982b-ab20ded7b1d5 service placement] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:03:48.675848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1264/2497] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:03:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:03:58.671719 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=27 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:04:13.757695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d36ed-ac13-4e12-ae4d-7a0b10a3a212 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:04:13.776012 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d36ed-ac13-4e12-ae4d-7a0b10a3a212 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:04:13.779679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd01a79-5b7f-4c43-9246-1297de7a6adb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:04:13.780297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd01a79-5b7f-4c43-9246-1297de7a6adb service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:04:13.780572 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd01a79-5b7f-4c43-9246-1297de7a6adb service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:04:13.780841 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd01a79-5b7f-4c43-9246-1297de7a6adb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:04:13.797432 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd01a79-5b7f-4c43-9246-1297de7a6adb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:04:13.798876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd01a79-5b7f-4c43-9246-1297de7a6adb service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:04:13.819842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1234/2498] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:04:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:04:23.817307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=18 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:04:59.664391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d04603-4f12-4419-982b-ab20ded7b1d5 service placement] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:04:59.683487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d04603-4f12-4419-982b-ab20ded7b1d5 service placement] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:04:59.687086 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9ab57-ce6b-4056-97f8-89ac1df0689f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:04:59.687668 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9ab57-ce6b-4056-97f8-89ac1df0689f service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:04:59.687948 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9ab57-ce6b-4056-97f8-89ac1df0689f service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:04:59.688219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9ab57-ce6b-4056-97f8-89ac1df0689f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:04:59.696416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b9ab57-ce6b-4056-97f8-89ac1df0689f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:04:59.697828 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b9ab57-ce6b-4056-97f8-89ac1df0689f service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:04:59.706602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1265/2499] 10.222.0.15 () {60 vars in 1396 bytes} [Mon Sep 5 15:04:59 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:05:00.773390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd01a79-5b7f-4c43-9246-1297de7a6adb service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:05:00.774821 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8dd01a79-5b7f-4c43-9246-1297de7a6adb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:05:00.816962 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd01a79-5b7f-4c43-9246-1297de7a6adb service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:05:00.820535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56644f-0396-4bba-acd2-2a3f40c66287 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:05:00.821131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56644f-0396-4bba-acd2-2a3f40c66287 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:05:00.821409 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56644f-0396-4bba-acd2-2a3f40c66287 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:05:00.821679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56644f-0396-4bba-acd2-2a3f40c66287 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:05:00.829799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da56644f-0396-4bba-acd2-2a3f40c66287 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:05:00.831268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da56644f-0396-4bba-acd2-2a3f40c66287 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:05:00.840713 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1235/2500] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:05:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:05:09.702606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=10 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:05:10.836055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=19 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:05:42.266650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9ab57-ce6b-4056-97f8-89ac1df0689f service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:05:42.320481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9ab57-ce6b-4056-97f8-89ac1df0689f service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:05:42.324102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca8266-4529-48bd-b73f-2944708da486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:05:42.324694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca8266-4529-48bd-b73f-2944708da486 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:05:42.324971 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca8266-4529-48bd-b73f-2944708da486 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:05:42.325241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca8266-4529-48bd-b73f-2944708da486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:05:42.327127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-2fca8266-4529-48bd-b73f-2944708da486 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:05:42.339542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca8266-4529-48bd-b73f-2944708da486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:05:42.339542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca8266-4529-48bd-b73f-2944708da486 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:05:42.346016 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1266/2501] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:05:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:05:52.341972 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=26 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:07:01.403358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56644f-0396-4bba-acd2-2a3f40c66287 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:07:01.412346 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56644f-0396-4bba-acd2-2a3f40c66287 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:07:01.416024 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bdefb-ac1d-4640-a5f1-fde6df3e58b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:07:01.416621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bdefb-ac1d-4640-a5f1-fde6df3e58b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:07:01.416899 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bdefb-ac1d-4640-a5f1-fde6df3e58b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:07:01.417198 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bdefb-ac1d-4640-a5f1-fde6df3e58b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:07:01.425650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025bdefb-ac1d-4640-a5f1-fde6df3e58b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:07:01.427774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025bdefb-ac1d-4640-a5f1-fde6df3e58b3 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:07:01.436708 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1236/2502] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:07:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:07:11.432629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=6 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:07:17.252350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca8266-4529-48bd-b73f-2944708da486 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:07:17.262115 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca8266-4529-48bd-b73f-2944708da486 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:07:17.265630 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09842820-2af0-4ae5-9ea7-2a8087a85cf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:07:17.266213 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09842820-2af0-4ae5-9ea7-2a8087a85cf0 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:07:17.266503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09842820-2af0-4ae5-9ea7-2a8087a85cf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:07:17.266838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09842820-2af0-4ae5-9ea7-2a8087a85cf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:07:17.268630 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-09842820-2af0-4ae5-9ea7-2a8087a85cf0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:07:17.349148 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09842820-2af0-4ae5-9ea7-2a8087a85cf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:07:17.350557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09842820-2af0-4ae5-9ea7-2a8087a85cf0 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:07:17.359568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1267/2503] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:07:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 15:07:17.440472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bdefb-ac1d-4640-a5f1-fde6df3e58b3 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:07:17.449889 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bdefb-ac1d-4640-a5f1-fde6df3e58b3 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:07:17.457417 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6ce60-b8f8-4ea1-b0b6-9bb682c8621d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:07:17.461556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6ce60-b8f8-4ea1-b0b6-9bb682c8621d service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:07:17.461556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6ce60-b8f8-4ea1-b0b6-9bb682c8621d service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:07:17.461556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6ce60-b8f8-4ea1-b0b6-9bb682c8621d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:07:17.461556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-feb6ce60-b8f8-4ea1-b0b6-9bb682c8621d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:07:17.539930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6ce60-b8f8-4ea1-b0b6-9bb682c8621d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:07:17.541426 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6ce60-b8f8-4ea1-b0b6-9bb682c8621d service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:07:17.559881 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1237/2504] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:07:17 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:07:27.355461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=37 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:07:27.557269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=41 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:08:54.565921 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09842820-2af0-4ae5-9ea7-2a8087a85cf0 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:08:54.570005 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-09842820-2af0-4ae5-9ea7-2a8087a85cf0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:08:54.646066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09842820-2af0-4ae5-9ea7-2a8087a85cf0 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:08:54.651029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccd6e8-9a05-4023-ad21-ea1f838df9d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:08:54.651627 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccd6e8-9a05-4023-ad21-ea1f838df9d1 service placement] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:08:54.651907 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccd6e8-9a05-4023-ad21-ea1f838df9d1 service placement] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:08:54.652180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccd6e8-9a05-4023-ad21-ea1f838df9d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:08:54.654479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-6dccd6e8-9a05-4023-ad21-ea1f838df9d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:08:54.665450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dccd6e8-9a05-4023-ad21-ea1f838df9d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:08:54.667059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dccd6e8-9a05-4023-ad21-ea1f838df9d1 service placement] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:08:54.677177 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1268/2505] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:08:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:09:04.673102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=37 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:09:18.904937 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6ce60-b8f8-4ea1-b0b6-9bb682c8621d service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:09:18.906577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-feb6ce60-b8f8-4ea1-b0b6-9bb682c8621d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:09:18.917846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6ce60-b8f8-4ea1-b0b6-9bb682c8621d service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:09:18.921364 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de9a3d-0b6d-4b55-8943-4f167da165c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:09:18.922501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de9a3d-0b6d-4b55-8943-4f167da165c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:09:18.922775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de9a3d-0b6d-4b55-8943-4f167da165c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:09:18.923072 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de9a3d-0b6d-4b55-8943-4f167da165c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:09:18.924802 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-41de9a3d-0b6d-4b55-8943-4f167da165c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:09:19.005961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de9a3d-0b6d-4b55-8943-4f167da165c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:09:19.007437 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de9a3d-0b6d-4b55-8943-4f167da165c2 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:09:19.018102 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1238/2506] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:09:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:09:20.207398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccd6e8-9a05-4023-ad21-ea1f838df9d1 service placement] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:09:20.216906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccd6e8-9a05-4023-ad21-ea1f838df9d1 service placement] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:09:20.220539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff00ae-c816-421c-be13-95f109e78c1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:09:20.221135 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff00ae-c816-421c-be13-95f109e78c1a service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:09:20.221415 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff00ae-c816-421c-be13-95f109e78c1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:09:20.221684 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff00ae-c816-421c-be13-95f109e78c1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:09:20.223530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f9ff00ae-c816-421c-be13-95f109e78c1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:09:20.237034 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff00ae-c816-421c-be13-95f109e78c1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:09:20.238457 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff00ae-c816-421c-be13-95f109e78c1a service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:09:20.249430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1269/2507] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:09:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:09:20.659765 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de9a3d-0b6d-4b55-8943-4f167da165c2 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:09:20.665340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-41de9a3d-0b6d-4b55-8943-4f167da165c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:09:20.676430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de9a3d-0b6d-4b55-8943-4f167da165c2 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:09:20.683267 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9670ab-53df-41bf-860a-1a3da1158b1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:09:20.683884 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9670ab-53df-41bf-860a-1a3da1158b1d service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:09:20.684339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9670ab-53df-41bf-860a-1a3da1158b1d service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:09:20.684611 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9670ab-53df-41bf-860a-1a3da1158b1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:09:20.695947 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9670ab-53df-41bf-860a-1a3da1158b1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:09:20.697778 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9670ab-53df-41bf-860a-1a3da1158b1d service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:09:20.710518 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1239/2508] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:09:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:09:30.244650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=6 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:09:30.705327 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=45 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:10:47.271328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff00ae-c816-421c-be13-95f109e78c1a service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:10:47.279961 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff00ae-c816-421c-be13-95f109e78c1a service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:10:47.283831 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d44394-142f-4501-ad6c-df28ca865abb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:10:47.284428 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d44394-142f-4501-ad6c-df28ca865abb service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:10:47.284709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d44394-142f-4501-ad6c-df28ca865abb service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:10:47.284979 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d44394-142f-4501-ad6c-df28ca865abb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:10:47.292570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d44394-142f-4501-ad6c-df28ca865abb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:10:47.293988 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d44394-142f-4501-ad6c-df28ca865abb service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:10:47.301693 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1270/2509] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:10:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:10:57.298269 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=6 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:11:01.897393 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9670ab-53df-41bf-860a-1a3da1158b1d service nova] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:11:01.951257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9670ab-53df-41bf-860a-1a3da1158b1d service nova] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:11:01.954843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed2c80-521f-4091-a66c-2e330e62585b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:11:01.955445 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed2c80-521f-4091-a66c-2e330e62585b service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:11:01.955726 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed2c80-521f-4091-a66c-2e330e62585b service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:11:01.956001 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed2c80-521f-4091-a66c-2e330e62585b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:11:01.957799 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-54ed2c80-521f-4091-a66c-2e330e62585b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:11:02.034049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed2c80-521f-4091-a66c-2e330e62585b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:11:02.035518 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed2c80-521f-4091-a66c-2e330e62585b service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:11:02.053408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1240/2510] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:11:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:11:02.115677 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d44394-142f-4501-ad6c-df28ca865abb service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:11:02.125568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d44394-142f-4501-ad6c-df28ca865abb service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:11:02.129404 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef27992-6b1b-43be-8a59-107a71db5448 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:11:02.130002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef27992-6b1b-43be-8a59-107a71db5448 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:11:02.130284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef27992-6b1b-43be-8a59-107a71db5448 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:11:02.130562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef27992-6b1b-43be-8a59-107a71db5448 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:11:02.132415 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5ef27992-6b1b-43be-8a59-107a71db5448 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:11:02.209844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef27992-6b1b-43be-8a59-107a71db5448 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:11:02.211218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef27992-6b1b-43be-8a59-107a71db5448 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:11:02.229189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1271/2511] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:11:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:11:12.050960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=61 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:11:12.226626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=41 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:13:02.488141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed2c80-521f-4091-a66c-2e330e62585b service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:02.496773 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed2c80-521f-4091-a66c-2e330e62585b service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:02.501045 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb0509-663b-4aa4-9d4a-5001dd527783 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:02.501643 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb0509-663b-4aa4-9d4a-5001dd527783 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:02.501924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb0509-663b-4aa4-9d4a-5001dd527783 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:02.502197 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb0509-663b-4aa4-9d4a-5001dd527783 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:02.512175 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb0509-663b-4aa4-9d4a-5001dd527783 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:02.513691 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb0509-663b-4aa4-9d4a-5001dd527783 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:02.533529 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1241/2512] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:13:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:13:02.568872 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef27992-6b1b-43be-8a59-107a71db5448 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:02.570622 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5ef27992-6b1b-43be-8a59-107a71db5448 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:02.582865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef27992-6b1b-43be-8a59-107a71db5448 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:02.586529 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f918a9-cd6a-4829-8b66-3a7206f05481 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:02.587240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f918a9-cd6a-4829-8b66-3a7206f05481 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:02.587526 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f918a9-cd6a-4829-8b66-3a7206f05481 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:02.587800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f918a9-cd6a-4829-8b66-3a7206f05481 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:02.596704 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f918a9-cd6a-4829-8b66-3a7206f05481 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:02.599773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f918a9-cd6a-4829-8b66-3a7206f05481 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:02.607573 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1272/2513] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:13:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:13:05.316028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb0509-663b-4aa4-9d4a-5001dd527783 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:05.321394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ebeb0509-663b-4aa4-9d4a-5001dd527783 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:05.343646 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb0509-663b-4aa4-9d4a-5001dd527783 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:05.355033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e4b7d-cbf1-4bc1-bb3f-48eb0f7d7de5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:05.355033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e4b7d-cbf1-4bc1-bb3f-48eb0f7d7de5 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:05.355033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e4b7d-cbf1-4bc1-bb3f-48eb0f7d7de5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:05.355033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e4b7d-cbf1-4bc1-bb3f-48eb0f7d7de5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:05.355498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-640e4b7d-cbf1-4bc1-bb3f-48eb0f7d7de5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:05.383471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e4b7d-cbf1-4bc1-bb3f-48eb0f7d7de5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:05.384905 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e4b7d-cbf1-4bc1-bb3f-48eb0f7d7de5 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:05.398440 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f918a9-cd6a-4829-8b66-3a7206f05481 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:05.405954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1242/2514] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:13:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:13:05.409964 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f918a9-cd6a-4829-8b66-3a7206f05481 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:05.413968 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4823ec-a428-4ac5-a8c4-173b20cc9d05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:05.414884 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4823ec-a428-4ac5-a8c4-173b20cc9d05 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:05.415163 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4823ec-a428-4ac5-a8c4-173b20cc9d05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:05.415436 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4823ec-a428-4ac5-a8c4-173b20cc9d05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:05.417349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-4e4823ec-a428-4ac5-a8c4-173b20cc9d05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:05.524111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4823ec-a428-4ac5-a8c4-173b20cc9d05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:05.525935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4823ec-a428-4ac5-a8c4-173b20cc9d05 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:05.549218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1273/2515] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:13:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:13:05.606463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b362d-0ef2-4d69-bcbd-608d69d60e54 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:05.606689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b362d-0ef2-4d69-bcbd-608d69d60e54 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:05.607017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b362d-0ef2-4d69-bcbd-608d69d60e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:05.680437 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-ab5b362d-0ef2-4d69-bcbd-608d69d60e54 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:13:05.682619 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ab5b362d-0ef2-4d69-bcbd-608d69d60e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:05.698023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1243/2516] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:13:05.706090 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6853c21-aab8-443c-86bc-4f4dcb7c5e7d None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:05.707147 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6853c21-aab8-443c-86bc-4f4dcb7c5e7d None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:05.707433 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6853c21-aab8-443c-86bc-4f4dcb7c5e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:05.734356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-f6853c21-aab8-443c-86bc-4f4dcb7c5e7d None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:13:05.735999 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f6853c21-aab8-443c-86bc-4f4dcb7c5e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:05.749409 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b362d-0ef2-4d69-bcbd-608d69d60e54 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:05.760576 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1274/2517] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:13:05.761719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b362d-0ef2-4d69-bcbd-608d69d60e54 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:05.768177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d318ae-c8a2-4b9c-a1e6-0bcb26e58aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:05.769473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d318ae-c8a2-4b9c-a1e6-0bcb26e58aee admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:05.769757 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d318ae-c8a2-4b9c-a1e6-0bcb26e58aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:05.770215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d318ae-c8a2-4b9c-a1e6-0bcb26e58aee admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:05.771078 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d318ae-c8a2-4b9c-a1e6-0bcb26e58aee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:13:05.771392 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d318ae-c8a2-4b9c-a1e6-0bcb26e58aee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:05.773044 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d318ae-c8a2-4b9c-a1e6-0bcb26e58aee admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:05.819906 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1244/2518] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 15:13:05.823881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6853c21-aab8-443c-86bc-4f4dcb7c5e7d None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:05.842125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6853c21-aab8-443c-86bc-4f4dcb7c5e7d None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:05.863955 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3020e5-c69f-4ebb-97bb-34ad8da46cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:05.864642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3020e5-c69f-4ebb-97bb-34ad8da46cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:05.865000 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3020e5-c69f-4ebb-97bb-34ad8da46cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:05.865341 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3020e5-c69f-4ebb-97bb-34ad8da46cd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:05.866576 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3020e5-c69f-4ebb-97bb-34ad8da46cd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:05.870984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3020e5-c69f-4ebb-97bb-34ad8da46cd5 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:05.989279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1275/2519] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:13:05 2022] POST /identity/v3/projects => generated 397 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:13:05.993645 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d318ae-c8a2-4b9c-a1e6-0bcb26e58aee admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:06.013969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d318ae-c8a2-4b9c-a1e6-0bcb26e58aee admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:06.084369 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a5c59-cb6c-4f48-9a97-f5e9365e2d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:06.085140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a5c59-cb6c-4f48-9a97-f5e9365e2d88 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:06.085520 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a5c59-cb6c-4f48-9a97-f5e9365e2d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:06.085983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a5c59-cb6c-4f48-9a97-f5e9365e2d88 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:06.086743 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a5c59-cb6c-4f48-9a97-f5e9365e2d88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:06.088816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a5c59-cb6c-4f48-9a97-f5e9365e2d88 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:06.179188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3020e5-c69f-4ebb-97bb-34ad8da46cd5 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:06.181092 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-af3020e5-c69f-4ebb-97bb-34ad8da46cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:06.259437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3020e5-c69f-4ebb-97bb-34ad8da46cd5 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:06.260261 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1245/2520] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:13:05 2022] POST /identity/v3/users => generated 299 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:13:06.265557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a137a2-6d2c-4524-989d-7eb599004b30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:06.266351 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a137a2-6d2c-4524-989d-7eb599004b30 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:06.266515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a137a2-6d2c-4524-989d-7eb599004b30 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:06.266745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a5c59-cb6c-4f48-9a97-f5e9365e2d88 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:06.266919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a137a2-6d2c-4524-989d-7eb599004b30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:06.268853 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-11a137a2-6d2c-4524-989d-7eb599004b30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:06.278988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a5c59-cb6c-4f48-9a97-f5e9365e2d88 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:06.283481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a137a2-6d2c-4524-989d-7eb599004b30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:06.289237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f728b5a-fd62-4f2e-bec0-e7c89002d653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:06.289237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f728b5a-fd62-4f2e-bec0-e7c89002d653 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:06.289237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f728b5a-fd62-4f2e-bec0-e7c89002d653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:06.289237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f728b5a-fd62-4f2e-bec0-e7c89002d653 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:06.289662 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a137a2-6d2c-4524-989d-7eb599004b30 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:06.289756 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f728b5a-fd62-4f2e-bec0-e7c89002d653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:06.289756 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f728b5a-fd62-4f2e-bec0-e7c89002d653 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:06.297226 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1276/2521] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:13:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 15:13:06.300986 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1246/2522] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:13:06.305541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a137a2-6d2c-4524-989d-7eb599004b30 service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:06.314721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:13:06.321914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a137a2-6d2c-4524-989d-7eb599004b30 service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:06.335347 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a059b-4d4f-443c-80f2-5424ef99c816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:06.336136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a059b-4d4f-443c-80f2-5424ef99c816 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:06.336450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a059b-4d4f-443c-80f2-5424ef99c816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:06.336803 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a059b-4d4f-443c-80f2-5424ef99c816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:06.338169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a059b-4d4f-443c-80f2-5424ef99c816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:06.340659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a059b-4d4f-443c-80f2-5424ef99c816 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:06.352033 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1277/2523] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 15:13:06.357060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f728b5a-fd62-4f2e-bec0-e7c89002d653 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:06.372975 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f728b5a-fd62-4f2e-bec0-e7c89002d653 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:06.381719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47ed3c-1991-4ed4-873b-d5822b52e531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:06.382541 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47ed3c-1991-4ed4-873b-d5822b52e531 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:06.382984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47ed3c-1991-4ed4-873b-d5822b52e531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:06.383358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47ed3c-1991-4ed4-873b-d5822b52e531 admin admin] PATH_INFO: `/v3/projects/6e167fbc00364410a7f2e0c93a3ab9d7/users/26e249e3b8394169ae786728dd62f95a/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:06.400765 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:13:06.407906 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47ed3c-1991-4ed4-873b-d5822b52e531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e167fbc00364410a7f2e0c93a3ab9d7, user_id=26e249e3b8394169ae786728dd62f95a, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:06.412470 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47ed3c-1991-4ed4-873b-d5822b52e531 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:06.497887 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1247/2524] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:13:06 2022] PUT /identity/v3/projects/6e167fbc00364410a7f2e0c93a3ab9d7/users/26e249e3b8394169ae786728dd62f95a/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:13:06.499846 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298835bf-7edf-40fe-b85e-de6d84ad8256 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:06.502481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298835bf-7edf-40fe-b85e-de6d84ad8256 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:06.503000 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298835bf-7edf-40fe-b85e-de6d84ad8256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:06.539395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-298835bf-7edf-40fe-b85e-de6d84ad8256 None None] MFA Rules not processed for user `26e249e3b8394169ae786728dd62f95a`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:13:06.540632 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-298835bf-7edf-40fe-b85e-de6d84ad8256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:06.655166 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1278/2525] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:13:06 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:13:06.671045 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47ed3c-1991-4ed4-873b-d5822b52e531 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:06.742952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47ed3c-1991-4ed4-873b-d5822b52e531 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:06.746664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13128163-d6ee-4865-8303-73a32edf5acb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:06.747419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13128163-d6ee-4865-8303-73a32edf5acb service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:06.747689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13128163-d6ee-4865-8303-73a32edf5acb service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:06.747964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13128163-d6ee-4865-8303-73a32edf5acb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:06.757075 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13128163-d6ee-4865-8303-73a32edf5acb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:06.758554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13128163-d6ee-4865-8303-73a32edf5acb service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:06.768735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1248/2526] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:13:06 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:13:15.463307 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298835bf-7edf-40fe-b85e-de6d84ad8256 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:15.533175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298835bf-7edf-40fe-b85e-de6d84ad8256 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:15.536862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1059d1-228a-4274-af01-d1eb7f2d6511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:15.537468 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1059d1-228a-4274-af01-d1eb7f2d6511 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:15.537745 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1059d1-228a-4274-af01-d1eb7f2d6511 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:15.538601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1059d1-228a-4274-af01-d1eb7f2d6511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:15.540440 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-fb1059d1-228a-4274-af01-d1eb7f2d6511 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:15.608237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1059d1-228a-4274-af01-d1eb7f2d6511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:15.609651 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1059d1-228a-4274-af01-d1eb7f2d6511 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:15.619268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1279/2527] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:13:16.777081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:13:17.835709 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13128163-d6ee-4865-8303-73a32edf5acb service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:17.849240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13128163-d6ee-4865-8303-73a32edf5acb service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:17.854449 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55e5be-2de3-470d-b46e-a8677947cde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:17.855122 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55e5be-2de3-470d-b46e-a8677947cde5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:17.855407 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55e5be-2de3-470d-b46e-a8677947cde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:17.855682 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55e5be-2de3-470d-b46e-a8677947cde5 admin admin] PATH_INFO: `/v3/users/26e249e3b8394169ae786728dd62f95a` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:17.856820 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55e5be-2de3-470d-b46e-a8677947cde5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26e249e3b8394169ae786728dd62f95a)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:17.858753 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55e5be-2de3-470d-b46e-a8677947cde5 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:18.072566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-6e55e5be-2de3-470d-b46e-a8677947cde5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26e249e3b8394169ae786728dd62f95a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:18.166039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-6e55e5be-2de3-470d-b46e-a8677947cde5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26e249e3b8394169ae786728dd62f95a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:18.166669 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-6e55e5be-2de3-470d-b46e-a8677947cde5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26e249e3b8394169ae786728dd62f95a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:18.180382 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-6e55e5be-2de3-470d-b46e-a8677947cde5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26e249e3b8394169ae786728dd62f95a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:18.187993 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-6e55e5be-2de3-470d-b46e-a8677947cde5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26e249e3b8394169ae786728dd62f95a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:18.194199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1249/2528] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:13:17 2022] DELETE /identity/v3/users/26e249e3b8394169ae786728dd62f95a => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:13:18.198745 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22faea4-7aed-42e4-b223-ba60b568a19d None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:18.199319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22faea4-7aed-42e4-b223-ba60b568a19d None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:18.199319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22faea4-7aed-42e4-b223-ba60b568a19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:18.225481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-c22faea4-7aed-42e4-b223-ba60b568a19d None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:13:18.227294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c22faea4-7aed-42e4-b223-ba60b568a19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:18.256802 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:13:18.291628 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1280/2529] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:13:18.302618 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55e5be-2de3-470d-b46e-a8677947cde5 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:18.302950 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-6e55e5be-2de3-470d-b46e-a8677947cde5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:18.372111 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55e5be-2de3-470d-b46e-a8677947cde5 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:18.376511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b6045-6fc5-49be-992f-59e43473bbac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:18.377146 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b6045-6fc5-49be-992f-59e43473bbac service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:18.377430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b6045-6fc5-49be-992f-59e43473bbac service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:18.377707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b6045-6fc5-49be-992f-59e43473bbac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:18.392012 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b6045-6fc5-49be-992f-59e43473bbac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:18.392012 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b6045-6fc5-49be-992f-59e43473bbac service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:18.402468 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1250/2530] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 15:13:18.926410 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22faea4-7aed-42e4-b223-ba60b568a19d None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:18.930411 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c22faea4-7aed-42e4-b223-ba60b568a19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:18.945274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22faea4-7aed-42e4-b223-ba60b568a19d None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:18.951363 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de314c1-20ed-48e5-8aeb-f6881ad0163b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:18.952149 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de314c1-20ed-48e5-8aeb-f6881ad0163b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:18.952538 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de314c1-20ed-48e5-8aeb-f6881ad0163b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:18.952921 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de314c1-20ed-48e5-8aeb-f6881ad0163b admin admin] PATH_INFO: `/v3/projects/6e167fbc00364410a7f2e0c93a3ab9d7` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:18.954176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de314c1-20ed-48e5-8aeb-f6881ad0163b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e167fbc00364410a7f2e0c93a3ab9d7)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:18.956336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de314c1-20ed-48e5-8aeb-f6881ad0163b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:19.152151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0de314c1-20ed-48e5-8aeb-f6881ad0163b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e167fbc00364410a7f2e0c93a3ab9d7', 'request_id': 'req-0de314c1-20ed-48e5-8aeb-f6881ad0163b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:19.158591 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0de314c1-20ed-48e5-8aeb-f6881ad0163b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e167fbc00364410a7f2e0c93a3ab9d7', 'request_id': 'req-0de314c1-20ed-48e5-8aeb-f6881ad0163b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:19.262435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0de314c1-20ed-48e5-8aeb-f6881ad0163b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e167fbc00364410a7f2e0c93a3ab9d7', 'request_id': 'req-0de314c1-20ed-48e5-8aeb-f6881ad0163b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:19.272699 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b6045-6fc5-49be-992f-59e43473bbac service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:19.274951 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-4a6b6045-6fc5-49be-992f-59e43473bbac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:19.277275 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0de314c1-20ed-48e5-8aeb-f6881ad0163b admin admin] The token cache is being invalidate because project 6e167fbc00364410a7f2e0c93a3ab9d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:13:19.277574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0de314c1-20ed-48e5-8aeb-f6881ad0163b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:19.278323 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-0de314c1-20ed-48e5-8aeb-f6881ad0163b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:19.279880 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1281/2531] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:13:18 2022] DELETE /identity/v3/projects/6e167fbc00364410a7f2e0c93a3ab9d7 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:13:19.357116 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b6045-6fc5-49be-992f-59e43473bbac service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:19.371662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:13:19.401812 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de314c1-20ed-48e5-8aeb-f6881ad0163b admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:19.404969 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0de314c1-20ed-48e5-8aeb-f6881ad0163b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:19.448132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4f473-da2e-4f67-825e-d7c7fc6297f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:19.449315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4f473-da2e-4f67-825e-d7c7fc6297f6 service placement] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:19.449991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4f473-da2e-4f67-825e-d7c7fc6297f6 service placement] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:19.450422 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4f473-da2e-4f67-825e-d7c7fc6297f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:19.453248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-2ea4f473-da2e-4f67-825e-d7c7fc6297f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:19.471675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4f473-da2e-4f67-825e-d7c7fc6297f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:19.474053 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4f473-da2e-4f67-825e-d7c7fc6297f6 service placement] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:19.488446 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1251/2532] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:13:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:13:19.493502 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de314c1-20ed-48e5-8aeb-f6881ad0163b admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:19.498069 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef2ed7-091f-49da-9c64-d775aecf1ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:19.498720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef2ed7-091f-49da-9c64-d775aecf1ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:19.499029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef2ed7-091f-49da-9c64-d775aecf1ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:19.499303 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef2ed7-091f-49da-9c64-d775aecf1ee9 admin admin] PATH_INFO: `/v3/users/9136f02e6c664f27b3f4fd35475f0eea` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:19.500560 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef2ed7-091f-49da-9c64-d775aecf1ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9136f02e6c664f27b3f4fd35475f0eea)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:19.503461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef2ed7-091f-49da-9c64-d775aecf1ee9 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:19.689657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-c9ef2ed7-091f-49da-9c64-d775aecf1ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9136f02e6c664f27b3f4fd35475f0eea'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:19.759661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-c9ef2ed7-091f-49da-9c64-d775aecf1ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9136f02e6c664f27b3f4fd35475f0eea'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:19.766446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:13:19.776376 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-c9ef2ed7-091f-49da-9c64-d775aecf1ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9136f02e6c664f27b3f4fd35475f0eea'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:19.779698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-c9ef2ed7-091f-49da-9c64-d775aecf1ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9136f02e6c664f27b3f4fd35475f0eea'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:19.784087 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-c9ef2ed7-091f-49da-9c64-d775aecf1ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9136f02e6c664f27b3f4fd35475f0eea'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:19.790809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1282/2533] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:13:19 2022] DELETE /identity/v3/users/9136f02e6c664f27b3f4fd35475f0eea => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:13:20.045648 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4f473-da2e-4f67-825e-d7c7fc6297f6 service placement] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:20.060762 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4f473-da2e-4f67-825e-d7c7fc6297f6 service placement] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:20.125271 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062710a-699a-4431-8e06-739bf9b1b8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:20.125930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062710a-699a-4431-8e06-739bf9b1b8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:20.126216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062710a-699a-4431-8e06-739bf9b1b8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:20.126493 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062710a-699a-4431-8e06-739bf9b1b8ed admin admin] PATH_INFO: `/v3/projects/2af009055e3b4bdda73a770242253b0d` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:20.127670 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062710a-699a-4431-8e06-739bf9b1b8ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=2af009055e3b4bdda73a770242253b0d)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:20.129577 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062710a-699a-4431-8e06-739bf9b1b8ed admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:20.222316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8062710a-699a-4431-8e06-739bf9b1b8ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2af009055e3b4bdda73a770242253b0d', 'request_id': 'req-8062710a-699a-4431-8e06-739bf9b1b8ed'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:20.228417 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8062710a-699a-4431-8e06-739bf9b1b8ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2af009055e3b4bdda73a770242253b0d', 'request_id': 'req-8062710a-699a-4431-8e06-739bf9b1b8ed'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:20.319472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8062710a-699a-4431-8e06-739bf9b1b8ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2af009055e3b4bdda73a770242253b0d', 'request_id': 'req-8062710a-699a-4431-8e06-739bf9b1b8ed'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:20.331098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8062710a-699a-4431-8e06-739bf9b1b8ed admin admin] The token cache is being invalidate because project 2af009055e3b4bdda73a770242253b0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:13:20.331387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8062710a-699a-4431-8e06-739bf9b1b8ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:20.332115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8062710a-699a-4431-8e06-739bf9b1b8ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:13:20.333543 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1252/2534] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:13:20 2022] DELETE /identity/v3/projects/2af009055e3b4bdda73a770242253b0d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:13:20.340116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455bf58-40fc-4199-81d5-090800d24c99 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:20.340581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455bf58-40fc-4199-81d5-090800d24c99 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:20.340705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455bf58-40fc-4199-81d5-090800d24c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:20.360194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-e455bf58-40fc-4199-81d5-090800d24c99 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:13:20.361416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e455bf58-40fc-4199-81d5-090800d24c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:20.419963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1283/2535] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:13:20.427644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae780df3-0217-44d7-904e-7acd7c375ab6 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:20.427936 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae780df3-0217-44d7-904e-7acd7c375ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:20.428215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae780df3-0217-44d7-904e-7acd7c375ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:20.450240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-ae780df3-0217-44d7-904e-7acd7c375ab6 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:13:20.451462 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ae780df3-0217-44d7-904e-7acd7c375ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:20.459447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1253/2536] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:13:20.463724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455bf58-40fc-4199-81d5-090800d24c99 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:20.474021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455bf58-40fc-4199-81d5-090800d24c99 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:20.478618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce03da-c60c-4fbe-8f57-9a9c28ae3bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:20.479252 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce03da-c60c-4fbe-8f57-9a9c28ae3bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:20.479540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce03da-c60c-4fbe-8f57-9a9c28ae3bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:20.479819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce03da-c60c-4fbe-8f57-9a9c28ae3bf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:20.480500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce03da-c60c-4fbe-8f57-9a9c28ae3bf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:13:20.480807 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce03da-c60c-4fbe-8f57-9a9c28ae3bf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:20.482422 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce03da-c60c-4fbe-8f57-9a9c28ae3bf2 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:20.494313 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1284/2537] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 15:13:20.498347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae780df3-0217-44d7-904e-7acd7c375ab6 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:20.508488 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae780df3-0217-44d7-904e-7acd7c375ab6 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:20.513157 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825542d3-bbdc-4edc-9707-ff9eafcee685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:20.513738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825542d3-bbdc-4edc-9707-ff9eafcee685 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:20.514013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825542d3-bbdc-4edc-9707-ff9eafcee685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:20.514279 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825542d3-bbdc-4edc-9707-ff9eafcee685 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:20.514946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825542d3-bbdc-4edc-9707-ff9eafcee685 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:20.516814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825542d3-bbdc-4edc-9707-ff9eafcee685 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:20.589602 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1254/2538] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:13:20 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:13:20.593709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce03da-c60c-4fbe-8f57-9a9c28ae3bf2 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:20.603777 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce03da-c60c-4fbe-8f57-9a9c28ae3bf2 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:20.659197 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cfb91-aae4-4714-9355-5d6bc1136b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:20.659857 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cfb91-aae4-4714-9355-5d6bc1136b9a admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:20.660141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cfb91-aae4-4714-9355-5d6bc1136b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:20.660412 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cfb91-aae4-4714-9355-5d6bc1136b9a admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:20.661035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cfb91-aae4-4714-9355-5d6bc1136b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:20.662990 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cfb91-aae4-4714-9355-5d6bc1136b9a admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:20.736373 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1285/2539] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:13:20 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:13:20.740502 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825542d3-bbdc-4edc-9707-ff9eafcee685 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:20.751307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825542d3-bbdc-4edc-9707-ff9eafcee685 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:20.756191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0d305-03be-49ca-92ab-fba0f31802de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:20.756784 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0d305-03be-49ca-92ab-fba0f31802de admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:20.757071 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0d305-03be-49ca-92ab-fba0f31802de admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:20.757345 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0d305-03be-49ca-92ab-fba0f31802de admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:20.758660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb0d305-03be-49ca-92ab-fba0f31802de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:20.760299 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb0d305-03be-49ca-92ab-fba0f31802de admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:20.762367 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:13:20.771155 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1255/2540] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:13:20.775073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cfb91-aae4-4714-9355-5d6bc1136b9a admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:20.785333 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cfb91-aae4-4714-9355-5d6bc1136b9a admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:20.790038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40382160-7918-407e-a75a-5f56cb01830b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:20.790630 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40382160-7918-407e-a75a-5f56cb01830b admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:20.790947 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40382160-7918-407e-a75a-5f56cb01830b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:20.795317 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40382160-7918-407e-a75a-5f56cb01830b admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:20.795317 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40382160-7918-407e-a75a-5f56cb01830b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:20.795317 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40382160-7918-407e-a75a-5f56cb01830b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:20.806142 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1286/2541] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 15:13:20.810054 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0d305-03be-49ca-92ab-fba0f31802de admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:20.821687 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0d305-03be-49ca-92ab-fba0f31802de admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:20.826558 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebcd34a-bd35-4e1c-8f39-ea604f77380a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:20.827660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcd34a-bd35-4e1c-8f39-ea604f77380a admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:20.827954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcd34a-bd35-4e1c-8f39-ea604f77380a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:20.828598 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcd34a-bd35-4e1c-8f39-ea604f77380a admin admin] PATH_INFO: `/v3/projects/7c858eb651e44140960444974d8bc4c4/users/3ad73a36c45a4763b00027457fb4117f/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:20.846644 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebcd34a-bd35-4e1c-8f39-ea604f77380a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c858eb651e44140960444974d8bc4c4, user_id=3ad73a36c45a4763b00027457fb4117f, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:20.849941 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebcd34a-bd35-4e1c-8f39-ea604f77380a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:20.914117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1256/2542] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:13:20 2022] PUT /identity/v3/projects/7c858eb651e44140960444974d8bc4c4/users/3ad73a36c45a4763b00027457fb4117f/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:13:20.919125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e323f7-29c9-444c-8b6d-586058cc7f2c None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:20.919438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e323f7-29c9-444c-8b6d-586058cc7f2c None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:20.919715 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e323f7-29c9-444c-8b6d-586058cc7f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:20.939970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-83e323f7-29c9-444c-8b6d-586058cc7f2c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:13:20.941163 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-83e323f7-29c9-444c-8b6d-586058cc7f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:20.977945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:13:20.999990 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1287/2543] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:13:21.010043 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebcd34a-bd35-4e1c-8f39-ea604f77380a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:21.011711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-eebcd34a-bd35-4e1c-8f39-ea604f77380a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:21.067865 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebcd34a-bd35-4e1c-8f39-ea604f77380a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:21.071684 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e573a2-02d6-4e11-9cb5-6fda89732d95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:21.072322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e573a2-02d6-4e11-9cb5-6fda89732d95 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:21.072636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e573a2-02d6-4e11-9cb5-6fda89732d95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:21.072904 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e573a2-02d6-4e11-9cb5-6fda89732d95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:21.081969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e573a2-02d6-4e11-9cb5-6fda89732d95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:21.083440 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e573a2-02d6-4e11-9cb5-6fda89732d95 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:21.093127 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1257/2544] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:13:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:13:26.187135 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacd290-7af6-4aa8-af1c-e6e6aa6c6d92 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:26.187824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacd290-7af6-4aa8-af1c-e6e6aa6c6d92 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:26.188115 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacd290-7af6-4aa8-af1c-e6e6aa6c6d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:26.239919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-faacd290-7af6-4aa8-af1c-e6e6aa6c6d92 None None] MFA Rules not processed for user `3ad73a36c45a4763b00027457fb4117f`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:13:26.241766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-faacd290-7af6-4aa8-af1c-e6e6aa6c6d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:26.318318 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1288/2545] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:13:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:13:26.334994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e573a2-02d6-4e11-9cb5-6fda89732d95 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:26.336984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-93e573a2-02d6-4e11-9cb5-6fda89732d95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:26.418567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e573a2-02d6-4e11-9cb5-6fda89732d95 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:26.424205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca192e-8c45-41e4-9dec-2cbb3cc46ab6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:26.424823 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca192e-8c45-41e4-9dec-2cbb3cc46ab6 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:26.425119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca192e-8c45-41e4-9dec-2cbb3cc46ab6 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:26.425372 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca192e-8c45-41e4-9dec-2cbb3cc46ab6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:26.433792 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca192e-8c45-41e4-9dec-2cbb3cc46ab6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:26.435412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca192e-8c45-41e4-9dec-2cbb3cc46ab6 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:26.444102 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1258/2546] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:13:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:13:28.777910 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad05414-fd2d-4f55-9546-84a816d7e2d8 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:28.778443 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad05414-fd2d-4f55-9546-84a816d7e2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:28.778443 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad05414-fd2d-4f55-9546-84a816d7e2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:28.799257 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-0ad05414-fd2d-4f55-9546-84a816d7e2d8 None None] MFA Rules not processed for user `3ad73a36c45a4763b00027457fb4117f`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:13:28.800483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0ad05414-fd2d-4f55-9546-84a816d7e2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:28.807541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1289/2547] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:13:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 15:13:29.136580 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca192e-8c45-41e4-9dec-2cbb3cc46ab6 service nova] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:29.145881 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca192e-8c45-41e4-9dec-2cbb3cc46ab6 service nova] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:29.150247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc87c3-1172-4d5f-afd9-e42dfd4a450e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:29.150917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc87c3-1172-4d5f-afd9-e42dfd4a450e service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:29.151203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc87c3-1172-4d5f-afd9-e42dfd4a450e service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:29.151473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc87c3-1172-4d5f-afd9-e42dfd4a450e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:29.153434 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c8fc87c3-1172-4d5f-afd9-e42dfd4a450e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:29.163238 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc87c3-1172-4d5f-afd9-e42dfd4a450e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:29.164726 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc87c3-1172-4d5f-afd9-e42dfd4a450e service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:29.173348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1259/2548] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:13:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:13:29.596296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad05414-fd2d-4f55-9546-84a816d7e2d8 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:29.598030 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0ad05414-fd2d-4f55-9546-84a816d7e2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:29.691003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad05414-fd2d-4f55-9546-84a816d7e2d8 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:29.696857 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263dc009-80b5-4a4f-af5b-16ec7c713a75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:29.700257 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dc009-80b5-4a4f-af5b-16ec7c713a75 service placement] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:29.700257 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dc009-80b5-4a4f-af5b-16ec7c713a75 service placement] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:29.700257 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dc009-80b5-4a4f-af5b-16ec7c713a75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:29.700257 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-263dc009-80b5-4a4f-af5b-16ec7c713a75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:29.712334 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263dc009-80b5-4a4f-af5b-16ec7c713a75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:29.713937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263dc009-80b5-4a4f-af5b-16ec7c713a75 service placement] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:29.726516 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1290/2549] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:13:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:13:39.169613 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=77,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:13:39.421031 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc87c3-1172-4d5f-afd9-e42dfd4a450e service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:39.423457 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc87c3-1172-4d5f-afd9-e42dfd4a450e service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:39.428959 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164a267-d607-499b-9b7a-603806869fcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:39.429556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164a267-d607-499b-9b7a-603806869fcd service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:39.429838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164a267-d607-499b-9b7a-603806869fcd service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:39.430159 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164a267-d607-499b-9b7a-603806869fcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:39.431902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9164a267-d607-499b-9b7a-603806869fcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:39.443914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164a267-d607-499b-9b7a-603806869fcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:39.445489 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164a267-d607-499b-9b7a-603806869fcd service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:39.455934 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1260/2550] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:13:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:13:39.720003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=75 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:13:49.451527 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=6 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:13:59.521146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263dc009-80b5-4a4f-af5b-16ec7c713a75 service placement] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:13:59.569204 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263dc009-80b5-4a4f-af5b-16ec7c713a75 service placement] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:13:59.574378 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d06849-bcc9-4818-ad87-f1a81659615c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:13:59.574903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d06849-bcc9-4818-ad87-f1a81659615c service placement] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:13:59.575183 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d06849-bcc9-4818-ad87-f1a81659615c service placement] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:13:59.575454 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d06849-bcc9-4818-ad87-f1a81659615c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:13:59.577140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-84d06849-bcc9-4818-ad87-f1a81659615c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:13:59.588907 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d06849-bcc9-4818-ad87-f1a81659615c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:13:59.590466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d06849-bcc9-4818-ad87-f1a81659615c service placement] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:13:59.600066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1291/2551] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:13:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:14:09.596023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=22 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:01.462861 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164a267-d607-499b-9b7a-603806869fcd service nova] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:01.465461 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9164a267-d607-499b-9b7a-603806869fcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:01.574843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164a267-d607-499b-9b7a-603806869fcd service nova] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:01.578467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ed83e-3564-4b37-8987-dbfba88a24e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:01.579200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ed83e-3564-4b37-8987-dbfba88a24e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:01.579485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ed83e-3564-4b37-8987-dbfba88a24e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:01.579758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ed83e-3564-4b37-8987-dbfba88a24e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:01.581594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e90ed83e-3564-4b37-8987-dbfba88a24e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:01.602294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ed83e-3564-4b37-8987-dbfba88a24e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:01.603763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ed83e-3564-4b37-8987-dbfba88a24e3 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:01.624812 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1261/2552] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:02.591963 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d06849-bcc9-4818-ad87-f1a81659615c service placement] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:02.593388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-84d06849-bcc9-4818-ad87-f1a81659615c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:02.634559 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d06849-bcc9-4818-ad87-f1a81659615c service placement] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:02.638270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bfc265-12f8-44ed-bf78-da9b8c979cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:02.638903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfc265-12f8-44ed-bf78-da9b8c979cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:02.639187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfc265-12f8-44ed-bf78-da9b8c979cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:02.639462 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfc265-12f8-44ed-bf78-da9b8c979cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:02.641253 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f8bfc265-12f8-44ed-bf78-da9b8c979cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:02.653887 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bfc265-12f8-44ed-bf78-da9b8c979cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:02.655390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bfc265-12f8-44ed-bf78-da9b8c979cc6 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:02.665290 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1292/2553] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:06.177938 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ed83e-3564-4b37-8987-dbfba88a24e3 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:06.190003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ed83e-3564-4b37-8987-dbfba88a24e3 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:06.195259 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80fcd4-97c5-4184-9394-4582ddd694d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:06.195864 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80fcd4-97c5-4184-9394-4582ddd694d3 service placement] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:06.196147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80fcd4-97c5-4184-9394-4582ddd694d3 service placement] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:06.196424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80fcd4-97c5-4184-9394-4582ddd694d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:06.198902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9a80fcd4-97c5-4184-9394-4582ddd694d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:06.210491 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80fcd4-97c5-4184-9394-4582ddd694d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:06.218923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80fcd4-97c5-4184-9394-4582ddd694d3 service placement] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:06.222283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1262/2554] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:06.308890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bfc265-12f8-44ed-bf78-da9b8c979cc6 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:06.310399 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f8bfc265-12f8-44ed-bf78-da9b8c979cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:06.325162 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bfc265-12f8-44ed-bf78-da9b8c979cc6 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:06.330736 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aacfa86-84e9-4289-a364-c23698ce9b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:06.331690 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacfa86-84e9-4289-a364-c23698ce9b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:06.332047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacfa86-84e9-4289-a364-c23698ce9b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:06.332395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacfa86-84e9-4289-a364-c23698ce9b2e admin admin] PATH_INFO: `/v3/users/c078e655c9454316aa7afe13a6b565e7` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:06.333712 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aacfa86-84e9-4289-a364-c23698ce9b2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c078e655c9454316aa7afe13a6b565e7)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:06.335806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aacfa86-84e9-4289-a364-c23698ce9b2e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:06.486736 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-9aacfa86-84e9-4289-a364-c23698ce9b2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c078e655c9454316aa7afe13a6b565e7'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:06.550500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-9aacfa86-84e9-4289-a364-c23698ce9b2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c078e655c9454316aa7afe13a6b565e7'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:06.563552 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-9aacfa86-84e9-4289-a364-c23698ce9b2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c078e655c9454316aa7afe13a6b565e7'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:06.564152 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-9aacfa86-84e9-4289-a364-c23698ce9b2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c078e655c9454316aa7afe13a6b565e7'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:06.571160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-9aacfa86-84e9-4289-a364-c23698ce9b2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c078e655c9454316aa7afe13a6b565e7'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:06.577064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1293/2555] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:15:06 2022] DELETE /identity/v3/users/c078e655c9454316aa7afe13a6b565e7 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:06.809353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80fcd4-97c5-4184-9394-4582ddd694d3 service placement] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:06.828126 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80fcd4-97c5-4184-9394-4582ddd694d3 service placement] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:06.885503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb9d68-d628-4508-9a79-8ddeefc4464a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:06.886131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb9d68-d628-4508-9a79-8ddeefc4464a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:06.886415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb9d68-d628-4508-9a79-8ddeefc4464a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:06.886689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb9d68-d628-4508-9a79-8ddeefc4464a admin admin] PATH_INFO: `/v3/projects/0f14cb621ef04aa3a17159e418d6c7aa` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:06.887827 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb9d68-d628-4508-9a79-8ddeefc4464a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f14cb621ef04aa3a17159e418d6c7aa)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:06.890123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb9d68-d628-4508-9a79-8ddeefc4464a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:06.986967 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:07.001277 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-c9eb9d68-d628-4508-9a79-8ddeefc4464a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f14cb621ef04aa3a17159e418d6c7aa', 'request_id': 'req-c9eb9d68-d628-4508-9a79-8ddeefc4464a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:07.006844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-c9eb9d68-d628-4508-9a79-8ddeefc4464a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f14cb621ef04aa3a17159e418d6c7aa', 'request_id': 'req-c9eb9d68-d628-4508-9a79-8ddeefc4464a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:07.055523 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-c9eb9d68-d628-4508-9a79-8ddeefc4464a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f14cb621ef04aa3a17159e418d6c7aa', 'request_id': 'req-c9eb9d68-d628-4508-9a79-8ddeefc4464a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:07.067474 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-c9eb9d68-d628-4508-9a79-8ddeefc4464a admin admin] The token cache is being invalidate because project 0f14cb621ef04aa3a17159e418d6c7aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:15:07.067474 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-c9eb9d68-d628-4508-9a79-8ddeefc4464a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:07.068033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-c9eb9d68-d628-4508-9a79-8ddeefc4464a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:07.069745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1263/2556] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:15:06 2022] DELETE /identity/v3/projects/0f14cb621ef04aa3a17159e418d6c7aa => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:07.076125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee34fd-b7e7-4191-855f-66661292e877 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:07.076421 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee34fd-b7e7-4191-855f-66661292e877 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:07.076698 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee34fd-b7e7-4191-855f-66661292e877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:07.101989 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-f7ee34fd-b7e7-4191-855f-66661292e877 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:07.101989 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f7ee34fd-b7e7-4191-855f-66661292e877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:07.157126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1294/2557] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 15:15:07.164697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdf7ab-b372-4414-8bfa-509a0062b1d8 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:07.164988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdf7ab-b372-4414-8bfa-509a0062b1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:07.165266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdf7ab-b372-4414-8bfa-509a0062b1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:07.185872 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-95cdf7ab-b372-4414-8bfa-509a0062b1d8 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:07.187088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-95cdf7ab-b372-4414-8bfa-509a0062b1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:07.194753 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1264/2558] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:07.199337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee34fd-b7e7-4191-855f-66661292e877 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:07.208832 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee34fd-b7e7-4191-855f-66661292e877 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:07.213437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a424f6-660c-436e-8ea7-719843ce3599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:07.214547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a424f6-660c-436e-8ea7-719843ce3599 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:07.216378 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a424f6-660c-436e-8ea7-719843ce3599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:07.216378 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a424f6-660c-436e-8ea7-719843ce3599 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:07.216378 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a424f6-660c-436e-8ea7-719843ce3599 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:15:07.216378 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a424f6-660c-436e-8ea7-719843ce3599 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:07.217719 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a424f6-660c-436e-8ea7-719843ce3599 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:07.229086 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1295/2559] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 15:15:07.233106 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cdf7ab-b372-4414-8bfa-509a0062b1d8 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:07.242524 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cdf7ab-b372-4414-8bfa-509a0062b1d8 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:07.247323 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfa610-b60d-408f-8157-231d50d27e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:07.247929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfa610-b60d-408f-8157-231d50d27e16 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:07.250909 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfa610-b60d-408f-8157-231d50d27e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:07.250909 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfa610-b60d-408f-8157-231d50d27e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:07.250909 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfa610-b60d-408f-8157-231d50d27e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:07.251487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfa610-b60d-408f-8157-231d50d27e16 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:07.329087 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1265/2560] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:15:07 2022] POST /identity/v3/projects => generated 363 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:15:07.333188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a424f6-660c-436e-8ea7-719843ce3599 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:07.343879 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a424f6-660c-436e-8ea7-719843ce3599 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:07.355927 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:07.400915 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bfae3-0ffc-4e9a-bc13-f920822acd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:07.401642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bfae3-0ffc-4e9a-bc13-f920822acd11 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:07.401928 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bfae3-0ffc-4e9a-bc13-f920822acd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:07.402202 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bfae3-0ffc-4e9a-bc13-f920822acd11 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:07.403437 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bfae3-0ffc-4e9a-bc13-f920822acd11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:07.405353 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bfae3-0ffc-4e9a-bc13-f920822acd11 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:07.528342 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1296/2561] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:15:07 2022] POST /identity/v3/users => generated 282 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:15:07.532281 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfa610-b60d-408f-8157-231d50d27e16 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:07.542325 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfa610-b60d-408f-8157-231d50d27e16 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:07.546863 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383edc35-6a8e-4f72-b5bf-b0688f104473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:07.547449 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383edc35-6a8e-4f72-b5bf-b0688f104473 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:07.547772 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383edc35-6a8e-4f72-b5bf-b0688f104473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:07.548035 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383edc35-6a8e-4f72-b5bf-b0688f104473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:07.548673 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383edc35-6a8e-4f72-b5bf-b0688f104473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:07.550286 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383edc35-6a8e-4f72-b5bf-b0688f104473 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:07.560496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1266/2562] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:15:07.564321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bfae3-0ffc-4e9a-bc13-f920822acd11 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:07.574364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bfae3-0ffc-4e9a-bc13-f920822acd11 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:07.579705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb73efd-175c-48d9-ae86-688ef0822fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:07.580258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb73efd-175c-48d9-ae86-688ef0822fdd admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:07.580520 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb73efd-175c-48d9-ae86-688ef0822fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:07.580793 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb73efd-175c-48d9-ae86-688ef0822fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:07.581425 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb73efd-175c-48d9-ae86-688ef0822fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:07.583044 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb73efd-175c-48d9-ae86-688ef0822fdd admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:07.594720 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1297/2563] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 15:15:07.598668 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383edc35-6a8e-4f72-b5bf-b0688f104473 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:07.608877 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383edc35-6a8e-4f72-b5bf-b0688f104473 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:07.613429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f648f83-490c-4d68-bf52-4feb9294c9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:07.614064 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f648f83-490c-4d68-bf52-4feb9294c9f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:07.614343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f648f83-490c-4d68-bf52-4feb9294c9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:07.614613 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f648f83-490c-4d68-bf52-4feb9294c9f9 admin admin] PATH_INFO: `/v3/projects/21bdf2e582bc4a718794624f2a3dc990/users/bd8aa1bda30f45f5af3ec2fd1455c58c/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:07.629740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f648f83-490c-4d68-bf52-4feb9294c9f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21bdf2e582bc4a718794624f2a3dc990, user_id=bd8aa1bda30f45f5af3ec2fd1455c58c, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:07.632675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f648f83-490c-4d68-bf52-4feb9294c9f9 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:07.699412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1267/2564] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:15:07 2022] PUT /identity/v3/projects/21bdf2e582bc4a718794624f2a3dc990/users/bd8aa1bda30f45f5af3ec2fd1455c58c/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:07.717213 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe721c1-a517-401f-b4fc-76e524866c5d None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:07.717982 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe721c1-a517-401f-b4fc-76e524866c5d None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:07.718615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe721c1-a517-401f-b4fc-76e524866c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:07.747077 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-ffe721c1-a517-401f-b4fc-76e524866c5d None None] MFA Rules not processed for user `bd8aa1bda30f45f5af3ec2fd1455c58c`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:07.748287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ffe721c1-a517-401f-b4fc-76e524866c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:07.812388 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1298/2565] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:07.817139 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f648f83-490c-4d68-bf52-4feb9294c9f9 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:07.877398 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f648f83-490c-4d68-bf52-4feb9294c9f9 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:07.882264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a02ab0-21f6-451e-a2a3-f7046e42e895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:07.882895 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a02ab0-21f6-451e-a2a3-f7046e42e895 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:07.883221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a02ab0-21f6-451e-a2a3-f7046e42e895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:07.883494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a02ab0-21f6-451e-a2a3-f7046e42e895 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:07.884127 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a02ab0-21f6-451e-a2a3-f7046e42e895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:07.886012 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a02ab0-21f6-451e-a2a3-f7046e42e895 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:07.981973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1268/2566] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:15:07 2022] POST /identity/v3/projects => generated 365 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:15:07.986116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe721c1-a517-401f-b4fc-76e524866c5d None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:07.996542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe721c1-a517-401f-b4fc-76e524866c5d None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:08.051491 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4f4f9-4624-4dcc-94d8-e6c1d1c3d771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:08.052084 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4f4f9-4624-4dcc-94d8-e6c1d1c3d771 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:08.052366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4f4f9-4624-4dcc-94d8-e6c1d1c3d771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:08.052641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4f4f9-4624-4dcc-94d8-e6c1d1c3d771 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:08.053253 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4f4f9-4624-4dcc-94d8-e6c1d1c3d771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:08.055215 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4f4f9-4624-4dcc-94d8-e6c1d1c3d771 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:08.127882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1299/2567] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:15:07 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:15:08.132065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a02ab0-21f6-451e-a2a3-f7046e42e895 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:08.142912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a02ab0-21f6-451e-a2a3-f7046e42e895 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:08.147471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f950742-0eab-4440-9a83-398bdc3692a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:08.148065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f950742-0eab-4440-9a83-398bdc3692a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:08.148344 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f950742-0eab-4440-9a83-398bdc3692a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:08.148615 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f950742-0eab-4440-9a83-398bdc3692a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:08.149244 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f950742-0eab-4440-9a83-398bdc3692a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:08.150861 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f950742-0eab-4440-9a83-398bdc3692a3 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:08.161698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1269/2568] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:15:08.165609 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4f4f9-4624-4dcc-94d8-e6c1d1c3d771 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:08.175356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4f4f9-4624-4dcc-94d8-e6c1d1c3d771 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:08.180350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e87d0-581c-472b-98fc-976a107802f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:08.181000 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e87d0-581c-472b-98fc-976a107802f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:08.181279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e87d0-581c-472b-98fc-976a107802f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:08.181549 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e87d0-581c-472b-98fc-976a107802f5 admin admin] PATH_INFO: `/v3/projects/190f1ecd08684c579db3263b2f40b5b8/users/c97548185e4d4a3fa0a410c7a99fc313/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:08.193845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:08.196751 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e87d0-581c-472b-98fc-976a107802f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=190f1ecd08684c579db3263b2f40b5b8, user_id=c97548185e4d4a3fa0a410c7a99fc313, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:08.199920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e87d0-581c-472b-98fc-976a107802f5 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:08.272456 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1300/2569] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:15:08 2022] PUT /identity/v3/projects/190f1ecd08684c579db3263b2f40b5b8/users/c97548185e4d4a3fa0a410c7a99fc313/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:08.276419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f950742-0eab-4440-9a83-398bdc3692a3 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:08.287080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f950742-0eab-4440-9a83-398bdc3692a3 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:08.310677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:08.342554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1350c-591c-43d2-a468-d357dacaac74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:08.343209 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1350c-591c-43d2-a468-d357dacaac74 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:08.343501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1350c-591c-43d2-a468-d357dacaac74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:08.343777 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1350c-591c-43d2-a468-d357dacaac74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:08.344413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1350c-591c-43d2-a468-d357dacaac74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:08.346005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1350c-591c-43d2-a468-d357dacaac74 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:08.355811 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1270/2570] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:15:08.359942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e87d0-581c-472b-98fc-976a107802f5 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:08.370276 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e87d0-581c-472b-98fc-976a107802f5 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:08.375466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e4892-a763-4200-8a62-de074a4f9763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:08.376115 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e4892-a763-4200-8a62-de074a4f9763 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:08.376395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e4892-a763-4200-8a62-de074a4f9763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:08.376667 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e4892-a763-4200-8a62-de074a4f9763 admin admin] PATH_INFO: `/v3/projects/190f1ecd08684c579db3263b2f40b5b8/users/c97548185e4d4a3fa0a410c7a99fc313/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:08.378626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e4892-a763-4200-8a62-de074a4f9763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=190f1ecd08684c579db3263b2f40b5b8, user_id=c97548185e4d4a3fa0a410c7a99fc313, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:08.381258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e4892-a763-4200-8a62-de074a4f9763 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:08.392418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1301/2571] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:15:08 2022] PUT /identity/v3/projects/190f1ecd08684c579db3263b2f40b5b8/users/c97548185e4d4a3fa0a410c7a99fc313/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:15:08.402668 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc66575-f7ff-42dc-a660-e7778cd6b403 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:08.403000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc66575-f7ff-42dc-a660-e7778cd6b403 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:08.403274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc66575-f7ff-42dc-a660-e7778cd6b403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:08.431074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-cbc66575-f7ff-42dc-a660-e7778cd6b403 None None] MFA Rules not processed for user `c97548185e4d4a3fa0a410c7a99fc313`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:08.432330 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-cbc66575-f7ff-42dc-a660-e7778cd6b403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:08.501296 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1271/2572] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:08.517197 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e4892-a763-4200-8a62-de074a4f9763 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:08.518907 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-405e4892-a763-4200-8a62-de074a4f9763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:08.574991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e4892-a763-4200-8a62-de074a4f9763 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:08.578585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281216e9-96ee-4d9d-ad76-cc24536848cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:08.579208 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281216e9-96ee-4d9d-ad76-cc24536848cb service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:08.579490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281216e9-96ee-4d9d-ad76-cc24536848cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:08.579761 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281216e9-96ee-4d9d-ad76-cc24536848cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:08.588471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281216e9-96ee-4d9d-ad76-cc24536848cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:08.589921 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281216e9-96ee-4d9d-ad76-cc24536848cb service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:08.599173 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1302/2573] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:08 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:08.787303 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc66575-f7ff-42dc-a660-e7778cd6b403 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:08.797264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc66575-f7ff-42dc-a660-e7778cd6b403 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:08.800751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037be02-69a8-4d72-8569-b558d1e6891e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:08.801335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037be02-69a8-4d72-8569-b558d1e6891e service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:08.801614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037be02-69a8-4d72-8569-b558d1e6891e service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:08.801885 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037be02-69a8-4d72-8569-b558d1e6891e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:08.856113 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037be02-69a8-4d72-8569-b558d1e6891e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:08.857620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037be02-69a8-4d72-8569-b558d1e6891e service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:08.866292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1272/2574] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:12.496059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281216e9-96ee-4d9d-ad76-cc24536848cb service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:12.497497 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-281216e9-96ee-4d9d-ad76-cc24536848cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:12.554260 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281216e9-96ee-4d9d-ad76-cc24536848cb service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:12.557899 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6387b29-4337-4c68-821b-7c0c82498f3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:12.558436 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6387b29-4337-4c68-821b-7c0c82498f3c service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:12.558711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6387b29-4337-4c68-821b-7c0c82498f3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:12.559008 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6387b29-4337-4c68-821b-7c0c82498f3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:12.560721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e6387b29-4337-4c68-821b-7c0c82498f3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:12.629300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6387b29-4337-4c68-821b-7c0c82498f3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:12.630736 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6387b29-4337-4c68-821b-7c0c82498f3c service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:12.642346 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1303/2575] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:18.862317 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=79 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:21.516620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037be02-69a8-4d72-8569-b558d1e6891e service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:21.578229 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037be02-69a8-4d72-8569-b558d1e6891e service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:21.582754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d9d34-5638-4ad7-99bb-95c430207d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:21.583443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d9d34-5638-4ad7-99bb-95c430207d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:21.583727 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d9d34-5638-4ad7-99bb-95c430207d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:21.584007 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d9d34-5638-4ad7-99bb-95c430207d21 admin admin] PATH_INFO: `/v3/users/bd8aa1bda30f45f5af3ec2fd1455c58c` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:21.585117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d9d34-5638-4ad7-99bb-95c430207d21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd8aa1bda30f45f5af3ec2fd1455c58c)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:21.587079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d9d34-5638-4ad7-99bb-95c430207d21 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:21.726730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-696d9d34-5638-4ad7-99bb-95c430207d21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd8aa1bda30f45f5af3ec2fd1455c58c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:21.775155 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-696d9d34-5638-4ad7-99bb-95c430207d21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd8aa1bda30f45f5af3ec2fd1455c58c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:21.775775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-696d9d34-5638-4ad7-99bb-95c430207d21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd8aa1bda30f45f5af3ec2fd1455c58c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:21.788387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-696d9d34-5638-4ad7-99bb-95c430207d21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd8aa1bda30f45f5af3ec2fd1455c58c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:21.795243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-696d9d34-5638-4ad7-99bb-95c430207d21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd8aa1bda30f45f5af3ec2fd1455c58c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:21.801041 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1273/2576] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:15:21 2022] DELETE /identity/v3/users/bd8aa1bda30f45f5af3ec2fd1455c58c => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:21.805501 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739d093-2c68-42e4-ae63-79adb5a87122 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:21.805995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739d093-2c68-42e4-ae63-79adb5a87122 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:21.806094 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739d093-2c68-42e4-ae63-79adb5a87122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:21.825243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-0739d093-2c68-42e4-ae63-79adb5a87122 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:21.826470 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0739d093-2c68-42e4-ae63-79adb5a87122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:21.843389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:21.886430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1304/2577] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 15:15:21.896639 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d9d34-5638-4ad7-99bb-95c430207d21 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:21.898067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-696d9d34-5638-4ad7-99bb-95c430207d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:21.954602 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d9d34-5638-4ad7-99bb-95c430207d21 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:21.958810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fc56a-280b-4783-879a-92fbdec79c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:21.959469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fc56a-280b-4783-879a-92fbdec79c71 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:21.959745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fc56a-280b-4783-879a-92fbdec79c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:21.960129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fc56a-280b-4783-879a-92fbdec79c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:21.969941 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fc56a-280b-4783-879a-92fbdec79c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:21.972028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fc56a-280b-4783-879a-92fbdec79c71 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:21.981731 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1274/2578] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:22.394775 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739d093-2c68-42e4-ae63-79adb5a87122 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:22.396280 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0739d093-2c68-42e4-ae63-79adb5a87122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:22.409551 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739d093-2c68-42e4-ae63-79adb5a87122 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:22.414962 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6e764-0258-47ab-99f7-c9f2f0127095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:22.415726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6e764-0258-47ab-99f7-c9f2f0127095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:22.415906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6e764-0258-47ab-99f7-c9f2f0127095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:22.416187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6e764-0258-47ab-99f7-c9f2f0127095 admin admin] PATH_INFO: `/v3/projects/21bdf2e582bc4a718794624f2a3dc990` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:22.417354 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa6e764-0258-47ab-99f7-c9f2f0127095 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21bdf2e582bc4a718794624f2a3dc990)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:22.419317 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa6e764-0258-47ab-99f7-c9f2f0127095 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:22.518459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-7fa6e764-0258-47ab-99f7-c9f2f0127095 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21bdf2e582bc4a718794624f2a3dc990', 'request_id': 'req-7fa6e764-0258-47ab-99f7-c9f2f0127095'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:22.524588 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-7fa6e764-0258-47ab-99f7-c9f2f0127095 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21bdf2e582bc4a718794624f2a3dc990', 'request_id': 'req-7fa6e764-0258-47ab-99f7-c9f2f0127095'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:22.578294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-7fa6e764-0258-47ab-99f7-c9f2f0127095 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21bdf2e582bc4a718794624f2a3dc990', 'request_id': 'req-7fa6e764-0258-47ab-99f7-c9f2f0127095'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:22.588936 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-7fa6e764-0258-47ab-99f7-c9f2f0127095 admin admin] The token cache is being invalidate because project 21bdf2e582bc4a718794624f2a3dc990 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:15:22.589237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-7fa6e764-0258-47ab-99f7-c9f2f0127095 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:22.589881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-7fa6e764-0258-47ab-99f7-c9f2f0127095 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:22.591386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1305/2579] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:15:22 2022] DELETE /identity/v3/projects/21bdf2e582bc4a718794624f2a3dc990 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:22.595527 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fc56a-280b-4783-879a-92fbdec79c71 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:22.605081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fc56a-280b-4783-879a-92fbdec79c71 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:22.623397 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:22.660780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aff7b1-bb66-4b2a-bafb-4eef38732047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:22.661593 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aff7b1-bb66-4b2a-bafb-4eef38732047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:22.661992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aff7b1-bb66-4b2a-bafb-4eef38732047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:22.662636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aff7b1-bb66-4b2a-bafb-4eef38732047 admin admin] PATH_INFO: `/v3/users/c97548185e4d4a3fa0a410c7a99fc313` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:22.663795 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aff7b1-bb66-4b2a-bafb-4eef38732047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c97548185e4d4a3fa0a410c7a99fc313)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:22.665718 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aff7b1-bb66-4b2a-bafb-4eef38732047 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:22.874853 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f6aff7b1-bb66-4b2a-bafb-4eef38732047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c97548185e4d4a3fa0a410c7a99fc313'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:22.952150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f6aff7b1-bb66-4b2a-bafb-4eef38732047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c97548185e4d4a3fa0a410c7a99fc313'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:22.952763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f6aff7b1-bb66-4b2a-bafb-4eef38732047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c97548185e4d4a3fa0a410c7a99fc313'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:22.965010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f6aff7b1-bb66-4b2a-bafb-4eef38732047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c97548185e4d4a3fa0a410c7a99fc313'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:22.972387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-f6aff7b1-bb66-4b2a-bafb-4eef38732047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c97548185e4d4a3fa0a410c7a99fc313'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:22.978265 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1275/2580] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:15:22 2022] DELETE /identity/v3/users/c97548185e4d4a3fa0a410c7a99fc313 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:23.482935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6e764-0258-47ab-99f7-c9f2f0127095 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:23.484486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7fa6e764-0258-47ab-99f7-c9f2f0127095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:23.503105 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6e764-0258-47ab-99f7-c9f2f0127095 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:23.557706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6829dc5-672f-4fe7-b041-a9fca25c4804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:23.558339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6829dc5-672f-4fe7-b041-a9fca25c4804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:23.558620 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6829dc5-672f-4fe7-b041-a9fca25c4804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:23.558922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6829dc5-672f-4fe7-b041-a9fca25c4804 admin admin] PATH_INFO: `/v3/projects/190f1ecd08684c579db3263b2f40b5b8` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:23.560057 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6829dc5-672f-4fe7-b041-a9fca25c4804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=190f1ecd08684c579db3263b2f40b5b8)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:23.561992 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6829dc5-672f-4fe7-b041-a9fca25c4804 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:23.692225 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:23.693649 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f6829dc5-672f-4fe7-b041-a9fca25c4804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '190f1ecd08684c579db3263b2f40b5b8', 'request_id': 'req-f6829dc5-672f-4fe7-b041-a9fca25c4804'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:23.699946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f6829dc5-672f-4fe7-b041-a9fca25c4804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '190f1ecd08684c579db3263b2f40b5b8', 'request_id': 'req-f6829dc5-672f-4fe7-b041-a9fca25c4804'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:23.761303 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f6829dc5-672f-4fe7-b041-a9fca25c4804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '190f1ecd08684c579db3263b2f40b5b8', 'request_id': 'req-f6829dc5-672f-4fe7-b041-a9fca25c4804'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:23.772783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f6829dc5-672f-4fe7-b041-a9fca25c4804 admin admin] The token cache is being invalidate because project 190f1ecd08684c579db3263b2f40b5b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:15:23.773073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f6829dc5-672f-4fe7-b041-a9fca25c4804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:23.773706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f6829dc5-672f-4fe7-b041-a9fca25c4804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:23.775206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1306/2581] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:15:23 2022] DELETE /identity/v3/projects/190f1ecd08684c579db3263b2f40b5b8 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:23.780030 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0ce12-620a-4f9f-8355-7f0fd08c6631 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:23.780487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0ce12-620a-4f9f-8355-7f0fd08c6631 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:23.780578 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0ce12-620a-4f9f-8355-7f0fd08c6631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:23.800781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-23d0ce12-620a-4f9f-8355-7f0fd08c6631 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:23.801986 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-23d0ce12-620a-4f9f-8355-7f0fd08c6631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:23.860740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1276/2582] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:23.868550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b580e74-b5d6-4e84-a8e4-46c30798c927 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:23.868909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b580e74-b5d6-4e84-a8e4-46c30798c927 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:23.869185 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b580e74-b5d6-4e84-a8e4-46c30798c927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:23.889571 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-9b580e74-b5d6-4e84-a8e4-46c30798c927 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:23.890812 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-9b580e74-b5d6-4e84-a8e4-46c30798c927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:23.899326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1307/2583] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:23.903267 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0ce12-620a-4f9f-8355-7f0fd08c6631 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:23.912738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0ce12-620a-4f9f-8355-7f0fd08c6631 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:23.917232 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1f60d-06a5-4d3e-bf2a-c103829ddcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:23.917850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1f60d-06a5-4d3e-bf2a-c103829ddcac admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:23.918132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1f60d-06a5-4d3e-bf2a-c103829ddcac admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:23.918406 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1f60d-06a5-4d3e-bf2a-c103829ddcac admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:23.919114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1f60d-06a5-4d3e-bf2a-c103829ddcac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:15:23.919418 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1f60d-06a5-4d3e-bf2a-c103829ddcac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:23.921023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1f60d-06a5-4d3e-bf2a-c103829ddcac admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:23.932764 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1277/2584] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 15:15:23.936869 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b580e74-b5d6-4e84-a8e4-46c30798c927 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:23.945937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b580e74-b5d6-4e84-a8e4-46c30798c927 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:23.950514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300777e-8d3c-4367-8d4f-e0b2cb3a6f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:23.951137 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300777e-8d3c-4367-8d4f-e0b2cb3a6f55 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:23.951417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300777e-8d3c-4367-8d4f-e0b2cb3a6f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:23.951689 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300777e-8d3c-4367-8d4f-e0b2cb3a6f55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:23.952309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300777e-8d3c-4367-8d4f-e0b2cb3a6f55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:23.954185 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300777e-8d3c-4367-8d4f-e0b2cb3a6f55 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:24.048194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1308/2585] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:15:23 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:15:24.052198 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1f60d-06a5-4d3e-bf2a-c103829ddcac admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:24.063037 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1f60d-06a5-4d3e-bf2a-c103829ddcac admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:24.088614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:24.119005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8800b6-4171-4d1e-9687-2f9a452148c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:24.119611 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8800b6-4171-4d1e-9687-2f9a452148c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:24.119890 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8800b6-4171-4d1e-9687-2f9a452148c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:24.120163 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8800b6-4171-4d1e-9687-2f9a452148c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:24.120792 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8800b6-4171-4d1e-9687-2f9a452148c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:24.122709 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8800b6-4171-4d1e-9687-2f9a452148c7 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:24.228099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1278/2586] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:15:24 2022] POST /identity/v3/users => generated 294 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:15:24.232241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300777e-8d3c-4367-8d4f-e0b2cb3a6f55 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:24.242852 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300777e-8d3c-4367-8d4f-e0b2cb3a6f55 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:24.247611 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d7839-17ab-453d-9ef3-e335b59a3701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:24.248213 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d7839-17ab-453d-9ef3-e335b59a3701 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:24.249674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d7839-17ab-453d-9ef3-e335b59a3701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:24.249674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d7839-17ab-453d-9ef3-e335b59a3701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:24.249674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d7839-17ab-453d-9ef3-e335b59a3701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:24.251143 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d7839-17ab-453d-9ef3-e335b59a3701 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:24.264640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1309/2587] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:15:24.265987 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8800b6-4171-4d1e-9687-2f9a452148c7 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:24.277677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8800b6-4171-4d1e-9687-2f9a452148c7 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:24.282475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792cc67-58d5-4077-936f-021fbd9d44d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:24.283671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792cc67-58d5-4077-936f-021fbd9d44d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:24.283971 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792cc67-58d5-4077-936f-021fbd9d44d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:24.284246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792cc67-58d5-4077-936f-021fbd9d44d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:24.284880 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792cc67-58d5-4077-936f-021fbd9d44d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:24.286491 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792cc67-58d5-4077-936f-021fbd9d44d7 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:24.297452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1279/2588] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:15:24.303294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d7839-17ab-453d-9ef3-e335b59a3701 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:24.335218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d7839-17ab-453d-9ef3-e335b59a3701 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:24.340542 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77829b38-ee78-4c88-a891-752ef002037f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:24.341287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77829b38-ee78-4c88-a891-752ef002037f admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:24.341606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77829b38-ee78-4c88-a891-752ef002037f admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:24.341916 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77829b38-ee78-4c88-a891-752ef002037f admin admin] PATH_INFO: `/v3/projects/75b4a17fcb7b41c39452d6db761fb056/users/db09d6efdc6143d6bd0ec6c96317803f/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:24.356513 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77829b38-ee78-4c88-a891-752ef002037f admin admin] RBAC: Authorizing `identity:create_grant(project_id=75b4a17fcb7b41c39452d6db761fb056, user_id=db09d6efdc6143d6bd0ec6c96317803f, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:24.359223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77829b38-ee78-4c88-a891-752ef002037f admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:24.439593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1310/2589] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:15:24 2022] PUT /identity/v3/projects/75b4a17fcb7b41c39452d6db761fb056/users/db09d6efdc6143d6bd0ec6c96317803f/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:24.451021 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d5383-b00f-42c1-bbc6-4988a6c52d38 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:24.451309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d5383-b00f-42c1-bbc6-4988a6c52d38 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:24.451585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d5383-b00f-42c1-bbc6-4988a6c52d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:24.481181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-663d5383-b00f-42c1-bbc6-4988a6c52d38 None None] MFA Rules not processed for user `db09d6efdc6143d6bd0ec6c96317803f`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:24.483006 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-663d5383-b00f-42c1-bbc6-4988a6c52d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:24.547355 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1280/2590] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:24.551762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77829b38-ee78-4c88-a891-752ef002037f admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:24.612026 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77829b38-ee78-4c88-a891-752ef002037f admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:24.616519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eeafc0-b0a0-4692-b4a6-e138627f5146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:24.617113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eeafc0-b0a0-4692-b4a6-e138627f5146 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:24.617390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eeafc0-b0a0-4692-b4a6-e138627f5146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:24.617658 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eeafc0-b0a0-4692-b4a6-e138627f5146 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:24.618268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eeafc0-b0a0-4692-b4a6-e138627f5146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:24.620180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eeafc0-b0a0-4692-b4a6-e138627f5146 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:24.702184 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1311/2591] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:15:24 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:15:24.706653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d5383-b00f-42c1-bbc6-4988a6c52d38 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:24.717096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d5383-b00f-42c1-bbc6-4988a6c52d38 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:24.772592 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd0c25-bd93-4de9-ab76-0a5646cdc9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:24.773192 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd0c25-bd93-4de9-ab76-0a5646cdc9cd admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:24.773473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd0c25-bd93-4de9-ab76-0a5646cdc9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:24.773750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd0c25-bd93-4de9-ab76-0a5646cdc9cd admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:24.774374 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd0c25-bd93-4de9-ab76-0a5646cdc9cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:24.776319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd0c25-bd93-4de9-ab76-0a5646cdc9cd admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:24.896053 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1281/2592] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:15:24 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:15:24.900049 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eeafc0-b0a0-4692-b4a6-e138627f5146 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:24.910532 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eeafc0-b0a0-4692-b4a6-e138627f5146 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:24.915286 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b499963-f85e-415a-9cb7-5411243122c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:24.921856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b499963-f85e-415a-9cb7-5411243122c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:24.921856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b499963-f85e-415a-9cb7-5411243122c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:24.921856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b499963-f85e-415a-9cb7-5411243122c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:24.921856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b499963-f85e-415a-9cb7-5411243122c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:24.921856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b499963-f85e-415a-9cb7-5411243122c6 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:24.929735 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1312/2593] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:15:24.933679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd0c25-bd93-4de9-ab76-0a5646cdc9cd admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:24.944830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd0c25-bd93-4de9-ab76-0a5646cdc9cd admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:24.949815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669d0eb-ce98-4264-8190-1cb1b164f4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:24.956328 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669d0eb-ce98-4264-8190-1cb1b164f4fc admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:24.956328 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669d0eb-ce98-4264-8190-1cb1b164f4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:24.956328 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669d0eb-ce98-4264-8190-1cb1b164f4fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:24.956328 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669d0eb-ce98-4264-8190-1cb1b164f4fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:24.956328 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669d0eb-ce98-4264-8190-1cb1b164f4fc admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:24.964292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1282/2594] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:15:24.968576 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b499963-f85e-415a-9cb7-5411243122c6 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:24.978438 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b499963-f85e-415a-9cb7-5411243122c6 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:24.982982 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98853eb6-fe8b-4d70-a50b-1d973747fb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:24.983623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98853eb6-fe8b-4d70-a50b-1d973747fb52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:24.983900 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98853eb6-fe8b-4d70-a50b-1d973747fb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:24.984172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98853eb6-fe8b-4d70-a50b-1d973747fb52 admin admin] PATH_INFO: `/v3/projects/f36f4e733142417686aa54080179af81/users/cda4e194cf214e478bad323d16d71e0b/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:24.998840 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98853eb6-fe8b-4d70-a50b-1d973747fb52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f36f4e733142417686aa54080179af81, user_id=cda4e194cf214e478bad323d16d71e0b, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:25.001450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98853eb6-fe8b-4d70-a50b-1d973747fb52 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:25.090238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1313/2595] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:15:24 2022] PUT /identity/v3/projects/f36f4e733142417686aa54080179af81/users/cda4e194cf214e478bad323d16d71e0b/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:15:25.107972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6792004-ebbd-42a4-98b4-2138816336b3 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:25.108368 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6792004-ebbd-42a4-98b4-2138816336b3 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:25.108697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6792004-ebbd-42a4-98b4-2138816336b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:25.137340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-a6792004-ebbd-42a4-98b4-2138816336b3 None None] MFA Rules not processed for user `cda4e194cf214e478bad323d16d71e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:25.138579 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a6792004-ebbd-42a4-98b4-2138816336b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:25.142524 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:25.201525 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1283/2596] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:25.205986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98853eb6-fe8b-4d70-a50b-1d973747fb52 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:25.231458 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:25.268328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98853eb6-fe8b-4d70-a50b-1d973747fb52 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:25.273336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fea4ca-f0cb-45b0-bb73-072ad5734e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:25.274013 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fea4ca-f0cb-45b0-bb73-072ad5734e52 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:25.274296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fea4ca-f0cb-45b0-bb73-072ad5734e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:25.274572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fea4ca-f0cb-45b0-bb73-072ad5734e52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:25.275231 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fea4ca-f0cb-45b0-bb73-072ad5734e52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:25.277213 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fea4ca-f0cb-45b0-bb73-072ad5734e52 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:25.374971 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1314/2597] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:15:25 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:15:25.379013 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6792004-ebbd-42a4-98b4-2138816336b3 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:25.389980 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6792004-ebbd-42a4-98b4-2138816336b3 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:25.445068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265bdc09-49ff-4562-b583-e426661d984c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:25.445668 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265bdc09-49ff-4562-b583-e426661d984c admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:25.445953 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265bdc09-49ff-4562-b583-e426661d984c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:25.446226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265bdc09-49ff-4562-b583-e426661d984c admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:25.446869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265bdc09-49ff-4562-b583-e426661d984c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:25.448774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265bdc09-49ff-4562-b583-e426661d984c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:25.532743 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1284/2598] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:15:25 2022] POST /identity/v3/users => generated 300 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:15:25.537436 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fea4ca-f0cb-45b0-bb73-072ad5734e52 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:25.548274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fea4ca-f0cb-45b0-bb73-072ad5734e52 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:25.553271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1cfff-b7e4-4fe1-be9c-ebbfe8162340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:25.553878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1cfff-b7e4-4fe1-be9c-ebbfe8162340 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:25.554158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1cfff-b7e4-4fe1-be9c-ebbfe8162340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:25.554432 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1cfff-b7e4-4fe1-be9c-ebbfe8162340 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:25.555147 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1cfff-b7e4-4fe1-be9c-ebbfe8162340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:25.559708 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1cfff-b7e4-4fe1-be9c-ebbfe8162340 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:25.568076 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1315/2599] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 15:15:25.572553 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265bdc09-49ff-4562-b583-e426661d984c admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:25.583347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265bdc09-49ff-4562-b583-e426661d984c admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:25.588131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d88c7-b54e-4934-b844-9786acc735ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:25.588776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d88c7-b54e-4934-b844-9786acc735ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:25.589055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d88c7-b54e-4934-b844-9786acc735ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:25.589328 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d88c7-b54e-4934-b844-9786acc735ee admin admin] PATH_INFO: `/v3/projects/a0c87d24d9ca4413aa0fe1f2fe45f636/users/9288b8366d8e498bbc90f3086327957b/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:25.604789 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d88c7-b54e-4934-b844-9786acc735ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0c87d24d9ca4413aa0fe1f2fe45f636, user_id=9288b8366d8e498bbc90f3086327957b, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:25.607497 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d88c7-b54e-4934-b844-9786acc735ee admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:25.693627 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1285/2600] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:15:25 2022] PUT /identity/v3/projects/a0c87d24d9ca4413aa0fe1f2fe45f636/users/9288b8366d8e498bbc90f3086327957b/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:25.697619 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1cfff-b7e4-4fe1-be9c-ebbfe8162340 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:25.708691 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1cfff-b7e4-4fe1-be9c-ebbfe8162340 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:25.766514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b4bba-6f0b-44d2-92eb-1580ee783269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:25.767675 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b4bba-6f0b-44d2-92eb-1580ee783269 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:25.768002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b4bba-6f0b-44d2-92eb-1580ee783269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:25.768277 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b4bba-6f0b-44d2-92eb-1580ee783269 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:25.768912 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b4bba-6f0b-44d2-92eb-1580ee783269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:25.770501 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b4bba-6f0b-44d2-92eb-1580ee783269 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:25.780061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1316/2601] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:15:25.785112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d88c7-b54e-4934-b844-9786acc735ee admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:25.795204 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d88c7-b54e-4934-b844-9786acc735ee admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:25.799737 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ccdba-254c-4bd7-b30a-91a6bfc30434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:25.800376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ccdba-254c-4bd7-b30a-91a6bfc30434 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:25.800653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ccdba-254c-4bd7-b30a-91a6bfc30434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:25.800927 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ccdba-254c-4bd7-b30a-91a6bfc30434 admin admin] PATH_INFO: `/v3/projects/a0c87d24d9ca4413aa0fe1f2fe45f636/users/9288b8366d8e498bbc90f3086327957b/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:25.802871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ccdba-254c-4bd7-b30a-91a6bfc30434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0c87d24d9ca4413aa0fe1f2fe45f636, user_id=9288b8366d8e498bbc90f3086327957b, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:25.805476 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ccdba-254c-4bd7-b30a-91a6bfc30434 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:25.815817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1286/2602] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:15:25 2022] PUT /identity/v3/projects/a0c87d24d9ca4413aa0fe1f2fe45f636/users/9288b8366d8e498bbc90f3086327957b/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:25.826894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0ced9-c03f-4262-8a9a-a512289fe7c4 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:25.827937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0ced9-c03f-4262-8a9a-a512289fe7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:25.827937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0ced9-c03f-4262-8a9a-a512289fe7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:25.855120 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-c7d0ced9-c03f-4262-8a9a-a512289fe7c4 None None] MFA Rules not processed for user `9288b8366d8e498bbc90f3086327957b`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:25.856325 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c7d0ced9-c03f-4262-8a9a-a512289fe7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:25.923932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1317/2603] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:25.934544 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ccdba-254c-4bd7-b30a-91a6bfc30434 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:25.936332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-cf6ccdba-254c-4bd7-b30a-91a6bfc30434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:25.993507 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ccdba-254c-4bd7-b30a-91a6bfc30434 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:25.997065 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae27fe-439a-4106-9f2c-1693a1bcafc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:25.997680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae27fe-439a-4106-9f2c-1693a1bcafc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:25.997968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae27fe-439a-4106-9f2c-1693a1bcafc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:25.998241 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae27fe-439a-4106-9f2c-1693a1bcafc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:26.007768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae27fe-439a-4106-9f2c-1693a1bcafc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:26.009198 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae27fe-439a-4106-9f2c-1693a1bcafc5 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:26.014023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:26.019328 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1287/2604] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:27.062202 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0ced9-c03f-4262-8a9a-a512289fe7c4 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:27.063734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c7d0ced9-c03f-4262-8a9a-a512289fe7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:27.076052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0ced9-c03f-4262-8a9a-a512289fe7c4 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:27.080376 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630598f2-009e-4e84-a85b-6cc729c491ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:27.080994 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630598f2-009e-4e84-a85b-6cc729c491ce service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:27.081277 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630598f2-009e-4e84-a85b-6cc729c491ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:27.081552 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630598f2-009e-4e84-a85b-6cc729c491ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:27.094418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:27.138168 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630598f2-009e-4e84-a85b-6cc729c491ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:27.139641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630598f2-009e-4e84-a85b-6cc729c491ce service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:27.149094 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1318/2605] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:27 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:32.007830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae27fe-439a-4106-9f2c-1693a1bcafc5 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:32.016409 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae27fe-439a-4106-9f2c-1693a1bcafc5 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:32.020136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623c1df-b872-413a-9c44-6b06d2902745 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:32.020731 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623c1df-b872-413a-9c44-6b06d2902745 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:32.021010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623c1df-b872-413a-9c44-6b06d2902745 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:32.021280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623c1df-b872-413a-9c44-6b06d2902745 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:32.077041 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623c1df-b872-413a-9c44-6b06d2902745 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:32.078572 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623c1df-b872-413a-9c44-6b06d2902745 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:32.087527 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1288/2606] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:32 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:35.736749 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630598f2-009e-4e84-a85b-6cc729c491ce service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:35.798487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630598f2-009e-4e84-a85b-6cc729c491ce service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:35.803083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f04ce7-aab9-4c76-bcab-5232ba9eb556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:35.803734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f04ce7-aab9-4c76-bcab-5232ba9eb556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:35.804014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f04ce7-aab9-4c76-bcab-5232ba9eb556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:35.804285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f04ce7-aab9-4c76-bcab-5232ba9eb556 admin admin] PATH_INFO: `/v3/users/db09d6efdc6143d6bd0ec6c96317803f` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:35.805445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f04ce7-aab9-4c76-bcab-5232ba9eb556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db09d6efdc6143d6bd0ec6c96317803f)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:35.807411 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f04ce7-aab9-4c76-bcab-5232ba9eb556 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:35.979415 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-25f04ce7-aab9-4c76-bcab-5232ba9eb556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db09d6efdc6143d6bd0ec6c96317803f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:36.060624 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-25f04ce7-aab9-4c76-bcab-5232ba9eb556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db09d6efdc6143d6bd0ec6c96317803f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:36.073487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-25f04ce7-aab9-4c76-bcab-5232ba9eb556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db09d6efdc6143d6bd0ec6c96317803f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:36.074144 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-25f04ce7-aab9-4c76-bcab-5232ba9eb556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db09d6efdc6143d6bd0ec6c96317803f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:36.083541 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-25f04ce7-aab9-4c76-bcab-5232ba9eb556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db09d6efdc6143d6bd0ec6c96317803f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:36.089415 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1319/2607] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:15:35 2022] DELETE /identity/v3/users/db09d6efdc6143d6bd0ec6c96317803f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:36.094099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9282ef-ac33-4676-b7ea-e553287d25a3 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:36.094556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9282ef-ac33-4676-b7ea-e553287d25a3 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:36.094643 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9282ef-ac33-4676-b7ea-e553287d25a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:36.113934 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-1a9282ef-ac33-4676-b7ea-e553287d25a3 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:36.115177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-1a9282ef-ac33-4676-b7ea-e553287d25a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:36.173188 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1289/2608] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:36.183471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f04ce7-aab9-4c76-bcab-5232ba9eb556 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:36.184886 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-25f04ce7-aab9-4c76-bcab-5232ba9eb556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:36.224200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:36.245898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f04ce7-aab9-4c76-bcab-5232ba9eb556 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:36.245898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2b930-e21d-45b6-872a-5531b3a2ebb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:36.245898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2b930-e21d-45b6-872a-5531b3a2ebb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:36.245898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2b930-e21d-45b6-872a-5531b3a2ebb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:36.246321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2b930-e21d-45b6-872a-5531b3a2ebb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:36.256297 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2b930-e21d-45b6-872a-5531b3a2ebb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:36.258037 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2b930-e21d-45b6-872a-5531b3a2ebb7 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:36.268122 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1320/2609] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:36.685466 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9282ef-ac33-4676-b7ea-e553287d25a3 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:36.686929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-1a9282ef-ac33-4676-b7ea-e553287d25a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:36.701216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9282ef-ac33-4676-b7ea-e553287d25a3 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:36.706313 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:36.707491 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:36.707760 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:36.708332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7 admin admin] PATH_INFO: `/v3/projects/75b4a17fcb7b41c39452d6db761fb056` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:36.709521 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75b4a17fcb7b41c39452d6db761fb056)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:36.711498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:36.818809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75b4a17fcb7b41c39452d6db761fb056', 'request_id': 'req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:36.824567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75b4a17fcb7b41c39452d6db761fb056', 'request_id': 'req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:36.898733 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75b4a17fcb7b41c39452d6db761fb056', 'request_id': 'req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:36.911401 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7 admin admin] The token cache is being invalidate because project 75b4a17fcb7b41c39452d6db761fb056 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:15:36.911695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:36.912343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:36.914339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1290/2610] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:15:36 2022] DELETE /identity/v3/projects/75b4a17fcb7b41c39452d6db761fb056 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:36.917901 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2b930-e21d-45b6-872a-5531b3a2ebb7 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:36.927677 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2b930-e21d-45b6-872a-5531b3a2ebb7 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:36.985011 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab9f9f-c623-466e-b1fe-68a56b4d6905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:36.985633 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab9f9f-c623-466e-b1fe-68a56b4d6905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:36.985914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab9f9f-c623-466e-b1fe-68a56b4d6905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:36.986186 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab9f9f-c623-466e-b1fe-68a56b4d6905 admin admin] PATH_INFO: `/v3/users/cda4e194cf214e478bad323d16d71e0b` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:36.987374 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab9f9f-c623-466e-b1fe-68a56b4d6905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cda4e194cf214e478bad323d16d71e0b)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:36.989272 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab9f9f-c623-466e-b1fe-68a56b4d6905 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:37.164813 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-dbab9f9f-c623-466e-b1fe-68a56b4d6905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cda4e194cf214e478bad323d16d71e0b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:37.244325 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-dbab9f9f-c623-466e-b1fe-68a56b4d6905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cda4e194cf214e478bad323d16d71e0b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:37.257861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-dbab9f9f-c623-466e-b1fe-68a56b4d6905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cda4e194cf214e478bad323d16d71e0b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:37.258499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-dbab9f9f-c623-466e-b1fe-68a56b4d6905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cda4e194cf214e478bad323d16d71e0b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:37.265276 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-dbab9f9f-c623-466e-b1fe-68a56b4d6905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cda4e194cf214e478bad323d16d71e0b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:37.271838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1321/2611] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:15:36 2022] DELETE /identity/v3/users/cda4e194cf214e478bad323d16d71e0b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:37.697335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:37.698848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:37.712256 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba17e5-e918-4d6b-a8af-ab59ac88e4a7 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:37.731244 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:37.771031 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb542f-47cd-4431-9c51-2f79c4099aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:37.771664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb542f-47cd-4431-9c51-2f79c4099aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:37.771945 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb542f-47cd-4431-9c51-2f79c4099aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:37.772218 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb542f-47cd-4431-9c51-2f79c4099aed admin admin] PATH_INFO: `/v3/projects/f36f4e733142417686aa54080179af81` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:37.773323 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb542f-47cd-4431-9c51-2f79c4099aed admin admin] RBAC: Authorizing `identity:delete_project(project_id=f36f4e733142417686aa54080179af81)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:37.775429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb542f-47cd-4431-9c51-2f79c4099aed admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:37.879144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8fdb542f-47cd-4431-9c51-2f79c4099aed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f36f4e733142417686aa54080179af81', 'request_id': 'req-8fdb542f-47cd-4431-9c51-2f79c4099aed'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:37.884842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8fdb542f-47cd-4431-9c51-2f79c4099aed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f36f4e733142417686aa54080179af81', 'request_id': 'req-8fdb542f-47cd-4431-9c51-2f79c4099aed'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:37.957858 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8fdb542f-47cd-4431-9c51-2f79c4099aed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f36f4e733142417686aa54080179af81', 'request_id': 'req-8fdb542f-47cd-4431-9c51-2f79c4099aed'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:37.969973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8fdb542f-47cd-4431-9c51-2f79c4099aed admin admin] The token cache is being invalidate because project f36f4e733142417686aa54080179af81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:15:37.970283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8fdb542f-47cd-4431-9c51-2f79c4099aed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:37.971461 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8fdb542f-47cd-4431-9c51-2f79c4099aed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:37.972980 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1291/2612] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:15:37 2022] DELETE /identity/v3/projects/f36f4e733142417686aa54080179af81 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:37.976957 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab9f9f-c623-466e-b1fe-68a56b4d6905 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:37.978387 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-dbab9f9f-c623-466e-b1fe-68a56b4d6905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:37.992411 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab9f9f-c623-466e-b1fe-68a56b4d6905 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:38.042956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:38.048654 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690c48c-7063-4cfc-92b4-72304f104f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:38.049927 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690c48c-7063-4cfc-92b4-72304f104f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:38.060287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690c48c-7063-4cfc-92b4-72304f104f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:38.060287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690c48c-7063-4cfc-92b4-72304f104f0e admin admin] PATH_INFO: `/v3/users/9288b8366d8e498bbc90f3086327957b` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:38.060287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690c48c-7063-4cfc-92b4-72304f104f0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9288b8366d8e498bbc90f3086327957b)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:38.060287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690c48c-7063-4cfc-92b4-72304f104f0e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:38.200320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-4690c48c-7063-4cfc-92b4-72304f104f0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9288b8366d8e498bbc90f3086327957b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:38.270002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-4690c48c-7063-4cfc-92b4-72304f104f0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9288b8366d8e498bbc90f3086327957b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:38.282193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-4690c48c-7063-4cfc-92b4-72304f104f0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9288b8366d8e498bbc90f3086327957b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:38.282844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-4690c48c-7063-4cfc-92b4-72304f104f0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9288b8366d8e498bbc90f3086327957b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:38.290227 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-4690c48c-7063-4cfc-92b4-72304f104f0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9288b8366d8e498bbc90f3086327957b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:38.296227 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1322/2613] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:15:37 2022] DELETE /identity/v3/users/9288b8366d8e498bbc90f3086327957b => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:38.728316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb542f-47cd-4431-9c51-2f79c4099aed admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:38.730024 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8fdb542f-47cd-4431-9c51-2f79c4099aed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:38.742664 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb542f-47cd-4431-9c51-2f79c4099aed admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:38.799987 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e6afa-10f8-46d0-b6c6-15d10731459c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:38.800614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e6afa-10f8-46d0-b6c6-15d10731459c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:38.800892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e6afa-10f8-46d0-b6c6-15d10731459c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:38.801163 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e6afa-10f8-46d0-b6c6-15d10731459c admin admin] PATH_INFO: `/v3/projects/a0c87d24d9ca4413aa0fe1f2fe45f636` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:38.802263 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e6afa-10f8-46d0-b6c6-15d10731459c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0c87d24d9ca4413aa0fe1f2fe45f636)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:38.804197 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e6afa-10f8-46d0-b6c6-15d10731459c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:38.923243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-db3e6afa-10f8-46d0-b6c6-15d10731459c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0c87d24d9ca4413aa0fe1f2fe45f636', 'request_id': 'req-db3e6afa-10f8-46d0-b6c6-15d10731459c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:38.927704 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:38.930027 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-db3e6afa-10f8-46d0-b6c6-15d10731459c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0c87d24d9ca4413aa0fe1f2fe45f636', 'request_id': 'req-db3e6afa-10f8-46d0-b6c6-15d10731459c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:39.032646 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-db3e6afa-10f8-46d0-b6c6-15d10731459c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0c87d24d9ca4413aa0fe1f2fe45f636', 'request_id': 'req-db3e6afa-10f8-46d0-b6c6-15d10731459c'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:39.045442 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-db3e6afa-10f8-46d0-b6c6-15d10731459c admin admin] The token cache is being invalidate because project a0c87d24d9ca4413aa0fe1f2fe45f636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:15:39.045734 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-db3e6afa-10f8-46d0-b6c6-15d10731459c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:39.046374 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-db3e6afa-10f8-46d0-b6c6-15d10731459c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:39.048140 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1292/2614] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:15:38 2022] DELETE /identity/v3/projects/a0c87d24d9ca4413aa0fe1f2fe45f636 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:39.054477 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc4f9f-ae66-4d2f-9e7e-0508718d685a None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:39.054948 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc4f9f-ae66-4d2f-9e7e-0508718d685a None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:39.055144 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc4f9f-ae66-4d2f-9e7e-0508718d685a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:39.075343 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-6fdc4f9f-ae66-4d2f-9e7e-0508718d685a None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:39.076556 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-6fdc4f9f-ae66-4d2f-9e7e-0508718d685a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:39.136364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1323/2615] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:39.144551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d5f69-801e-4c1d-83de-422f63af8f12 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:39.144842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d5f69-801e-4c1d-83de-422f63af8f12 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:39.145121 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d5f69-801e-4c1d-83de-422f63af8f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:39.164561 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-8f0d5f69-801e-4c1d-83de-422f63af8f12 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:39.165773 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8f0d5f69-801e-4c1d-83de-422f63af8f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:39.173682 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1293/2616] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:39.178398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc4f9f-ae66-4d2f-9e7e-0508718d685a None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:39.188405 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc4f9f-ae66-4d2f-9e7e-0508718d685a None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:39.193695 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d0c9a-c7c8-4d12-9d7b-943dcf362de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:39.194288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d0c9a-c7c8-4d12-9d7b-943dcf362de6 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:39.194571 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d0c9a-c7c8-4d12-9d7b-943dcf362de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:39.194883 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d0c9a-c7c8-4d12-9d7b-943dcf362de6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:39.195538 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d0c9a-c7c8-4d12-9d7b-943dcf362de6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:15:39.195841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d0c9a-c7c8-4d12-9d7b-943dcf362de6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:39.197443 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d0c9a-c7c8-4d12-9d7b-943dcf362de6 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:39.208820 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1324/2617] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 15:15:39.212943 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d5f69-801e-4c1d-83de-422f63af8f12 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:39.222260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d5f69-801e-4c1d-83de-422f63af8f12 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:39.230319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0ec16-f918-46e9-b487-4a83be8763ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:39.230319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0ec16-f918-46e9-b487-4a83be8763ec admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:39.230319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0ec16-f918-46e9-b487-4a83be8763ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:39.230319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0ec16-f918-46e9-b487-4a83be8763ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:39.230741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0ec16-f918-46e9-b487-4a83be8763ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:39.230741 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0ec16-f918-46e9-b487-4a83be8763ec admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:39.336689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1294/2618] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:15:39 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:15:39.340809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d0c9a-c7c8-4d12-9d7b-943dcf362de6 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:39.351369 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d0c9a-c7c8-4d12-9d7b-943dcf362de6 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:39.407653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40e5cf-516b-419e-89c9-b1f73eaa69de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:39.408259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40e5cf-516b-419e-89c9-b1f73eaa69de admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:39.408544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40e5cf-516b-419e-89c9-b1f73eaa69de admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:39.408822 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40e5cf-516b-419e-89c9-b1f73eaa69de admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:39.409946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f40e5cf-516b-419e-89c9-b1f73eaa69de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:39.411928 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f40e5cf-516b-419e-89c9-b1f73eaa69de admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:39.421660 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:39.533109 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1325/2619] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:15:39 2022] POST /identity/v3/users => generated 294 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:15:39.537785 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0ec16-f918-46e9-b487-4a83be8763ec admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:39.547862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0ec16-f918-46e9-b487-4a83be8763ec admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:39.553033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427538d-37f9-4828-bede-399b68d7910a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:39.553625 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427538d-37f9-4828-bede-399b68d7910a admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:39.553905 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427538d-37f9-4828-bede-399b68d7910a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:39.554178 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427538d-37f9-4828-bede-399b68d7910a admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:39.554846 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427538d-37f9-4828-bede-399b68d7910a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:39.556534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427538d-37f9-4828-bede-399b68d7910a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:39.566210 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1295/2620] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:15:39.570039 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40e5cf-516b-419e-89c9-b1f73eaa69de admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:39.579996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40e5cf-516b-419e-89c9-b1f73eaa69de admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:39.584635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e24479-4f2d-4535-bee5-89e71b54dc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:39.585223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e24479-4f2d-4535-bee5-89e71b54dc1d admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:39.585501 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e24479-4f2d-4535-bee5-89e71b54dc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:39.585768 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e24479-4f2d-4535-bee5-89e71b54dc1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:39.586396 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e24479-4f2d-4535-bee5-89e71b54dc1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:39.588096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e24479-4f2d-4535-bee5-89e71b54dc1d admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:39.600408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1326/2621] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 15:15:39.604387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427538d-37f9-4828-bede-399b68d7910a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:39.614935 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427538d-37f9-4828-bede-399b68d7910a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:39.619377 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9300f71-992c-412e-acdd-f8b064262a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:39.620018 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9300f71-992c-412e-acdd-f8b064262a49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:39.620299 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9300f71-992c-412e-acdd-f8b064262a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:39.620573 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9300f71-992c-412e-acdd-f8b064262a49 admin admin] PATH_INFO: `/v3/projects/2b8fc3d07b5a4d1d9a14fbfeb032a52d/users/0e48b258d8c04197843a6925dd9e822a/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:39.636500 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9300f71-992c-412e-acdd-f8b064262a49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b8fc3d07b5a4d1d9a14fbfeb032a52d, user_id=0e48b258d8c04197843a6925dd9e822a, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:39.639158 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9300f71-992c-412e-acdd-f8b064262a49 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:39.732260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1296/2622] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:15:39 2022] PUT /identity/v3/projects/2b8fc3d07b5a4d1d9a14fbfeb032a52d/users/0e48b258d8c04197843a6925dd9e822a/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:15:39.743641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8491547-e8f7-4839-ab45-6f05432ab7bd None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:39.743931 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8491547-e8f7-4839-ab45-6f05432ab7bd None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:39.744213 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8491547-e8f7-4839-ab45-6f05432ab7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:39.894194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-c8491547-e8f7-4839-ab45-6f05432ab7bd None None] MFA Rules not processed for user `0e48b258d8c04197843a6925dd9e822a`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:39.895345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c8491547-e8f7-4839-ab45-6f05432ab7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:39.956956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1327/2623] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:39.961521 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9300f71-992c-412e-acdd-f8b064262a49 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:40.021641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9300f71-992c-412e-acdd-f8b064262a49 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:40.026127 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecee9d-4af6-478e-9e14-501b206b2da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:40.026713 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecee9d-4af6-478e-9e14-501b206b2da6 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:40.027017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecee9d-4af6-478e-9e14-501b206b2da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:40.027287 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecee9d-4af6-478e-9e14-501b206b2da6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:40.027984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaecee9d-4af6-478e-9e14-501b206b2da6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:40.029884 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaecee9d-4af6-478e-9e14-501b206b2da6 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:40.113417 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1297/2624] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:15:39 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:15:40.117584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8491547-e8f7-4839-ab45-6f05432ab7bd None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:40.128725 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8491547-e8f7-4839-ab45-6f05432ab7bd None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:40.183140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfc374-a3c4-4fd0-93af-1d30cf6acac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:40.183675 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfc374-a3c4-4fd0-93af-1d30cf6acac9 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:40.183957 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfc374-a3c4-4fd0-93af-1d30cf6acac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:40.184228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfc374-a3c4-4fd0-93af-1d30cf6acac9 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:40.184856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcfc374-a3c4-4fd0-93af-1d30cf6acac9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:40.186814 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcfc374-a3c4-4fd0-93af-1d30cf6acac9 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:40.286861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1328/2625] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:15:40 2022] POST /identity/v3/users => generated 300 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:15:40.290807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecee9d-4af6-478e-9e14-501b206b2da6 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:40.301068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecee9d-4af6-478e-9e14-501b206b2da6 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:40.305620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4276ef-e533-48f0-9638-6fe8e139efa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:40.307370 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4276ef-e533-48f0-9638-6fe8e139efa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:40.307370 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4276ef-e533-48f0-9638-6fe8e139efa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:40.307370 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4276ef-e533-48f0-9638-6fe8e139efa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:40.307617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4276ef-e533-48f0-9638-6fe8e139efa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:40.309191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4276ef-e533-48f0-9638-6fe8e139efa4 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:40.319971 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1298/2626] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:15:40.323861 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfc374-a3c4-4fd0-93af-1d30cf6acac9 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:40.334219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfc374-a3c4-4fd0-93af-1d30cf6acac9 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:40.338744 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08e419-cfbc-4ef2-9e10-9149695d9623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:40.339398 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08e419-cfbc-4ef2-9e10-9149695d9623 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:40.339674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08e419-cfbc-4ef2-9e10-9149695d9623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:40.339961 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08e419-cfbc-4ef2-9e10-9149695d9623 admin admin] PATH_INFO: `/v3/projects/ad5aa8b4b6fd495d82f12278229589dd/users/524de55e6da94d2799da3d2d883d8e24/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:40.354062 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08e419-cfbc-4ef2-9e10-9149695d9623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad5aa8b4b6fd495d82f12278229589dd, user_id=524de55e6da94d2799da3d2d883d8e24, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:40.356683 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08e419-cfbc-4ef2-9e10-9149695d9623 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:40.430092 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1329/2627] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:15:40 2022] PUT /identity/v3/projects/ad5aa8b4b6fd495d82f12278229589dd/users/524de55e6da94d2799da3d2d883d8e24/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:15:40.433939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4276ef-e533-48f0-9638-6fe8e139efa4 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:40.444272 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4276ef-e533-48f0-9638-6fe8e139efa4 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:40.490329 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:40.498698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658ad8b-edb4-48f5-a3dc-14c1c08cba9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:40.499347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658ad8b-edb4-48f5-a3dc-14c1c08cba9b admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:40.499631 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658ad8b-edb4-48f5-a3dc-14c1c08cba9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:40.499903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658ad8b-edb4-48f5-a3dc-14c1c08cba9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:40.500536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658ad8b-edb4-48f5-a3dc-14c1c08cba9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:40.502134 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658ad8b-edb4-48f5-a3dc-14c1c08cba9b admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:40.512392 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1299/2628] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:15:40.516554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08e419-cfbc-4ef2-9e10-9149695d9623 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:40.526697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08e419-cfbc-4ef2-9e10-9149695d9623 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:40.531288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c15582-ccb8-420f-ad37-15752daee929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:40.531926 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c15582-ccb8-420f-ad37-15752daee929 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:40.532202 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c15582-ccb8-420f-ad37-15752daee929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:40.532475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c15582-ccb8-420f-ad37-15752daee929 admin admin] PATH_INFO: `/v3/projects/ad5aa8b4b6fd495d82f12278229589dd/users/524de55e6da94d2799da3d2d883d8e24/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:40.534350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c15582-ccb8-420f-ad37-15752daee929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad5aa8b4b6fd495d82f12278229589dd, user_id=524de55e6da94d2799da3d2d883d8e24, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:40.537030 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c15582-ccb8-420f-ad37-15752daee929 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:40.546997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1330/2629] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:15:40 2022] PUT /identity/v3/projects/ad5aa8b4b6fd495d82f12278229589dd/users/524de55e6da94d2799da3d2d883d8e24/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:40.558332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f56c15-2df6-4fc9-9406-2f373edda6aa None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:40.558616 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f56c15-2df6-4fc9-9406-2f373edda6aa None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:40.558928 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f56c15-2df6-4fc9-9406-2f373edda6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:40.587154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-26f56c15-2df6-4fc9-9406-2f373edda6aa None None] MFA Rules not processed for user `524de55e6da94d2799da3d2d883d8e24`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:40.588350 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-26f56c15-2df6-4fc9-9406-2f373edda6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:40.653804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1300/2630] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 15:15:40.664529 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c15582-ccb8-420f-ad37-15752daee929 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:40.666325 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a1c15582-ccb8-420f-ad37-15752daee929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:40.694458 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:40.722268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c15582-ccb8-420f-ad37-15752daee929 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:40.725829 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8286b9d-add1-4ddd-89ab-8ae51a15f085 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:40.726418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8286b9d-add1-4ddd-89ab-8ae51a15f085 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:40.726694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8286b9d-add1-4ddd-89ab-8ae51a15f085 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:40.726998 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8286b9d-add1-4ddd-89ab-8ae51a15f085 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:40.736121 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8286b9d-add1-4ddd-89ab-8ae51a15f085 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:40.738012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8286b9d-add1-4ddd-89ab-8ae51a15f085 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:40.747121 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1331/2631] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:40 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:40.889920 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f56c15-2df6-4fc9-9406-2f373edda6aa None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:40.950482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f56c15-2df6-4fc9-9406-2f373edda6aa None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:40.954963 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c26c87-7252-4a64-aaa4-1b3830c866e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:40.955587 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c26c87-7252-4a64-aaa4-1b3830c866e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:40.955896 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c26c87-7252-4a64-aaa4-1b3830c866e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:40.956174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c26c87-7252-4a64-aaa4-1b3830c866e1 admin admin] PATH_INFO: `/v3/users/0e48b258d8c04197843a6925dd9e822a` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:40.957335 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c26c87-7252-4a64-aaa4-1b3830c866e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e48b258d8c04197843a6925dd9e822a)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:40.959295 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c26c87-7252-4a64-aaa4-1b3830c866e1 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:41.150878 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-93c26c87-7252-4a64-aaa4-1b3830c866e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e48b258d8c04197843a6925dd9e822a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:41.215543 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-93c26c87-7252-4a64-aaa4-1b3830c866e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e48b258d8c04197843a6925dd9e822a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:41.216096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-93c26c87-7252-4a64-aaa4-1b3830c866e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e48b258d8c04197843a6925dd9e822a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:41.229137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-93c26c87-7252-4a64-aaa4-1b3830c866e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e48b258d8c04197843a6925dd9e822a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:41.233528 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:41.236385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-93c26c87-7252-4a64-aaa4-1b3830c866e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e48b258d8c04197843a6925dd9e822a'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:41.242472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1301/2632] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:15:40 2022] DELETE /identity/v3/users/0e48b258d8c04197843a6925dd9e822a => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:41.246976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96eb25-f1e3-4f62-bb81-9582f608e20b None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:41.247423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96eb25-f1e3-4f62-bb81-9582f608e20b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:41.247507 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96eb25-f1e3-4f62-bb81-9582f608e20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:41.266635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-aa96eb25-f1e3-4f62-bb81-9582f608e20b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:41.267869 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-aa96eb25-f1e3-4f62-bb81-9582f608e20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:41.325067 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1332/2633] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 15:15:41.335214 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c26c87-7252-4a64-aaa4-1b3830c866e1 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:41.336606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-93c26c87-7252-4a64-aaa4-1b3830c866e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:41.392176 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c26c87-7252-4a64-aaa4-1b3830c866e1 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:41.396325 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6330f-6fed-4dd2-92d9-e80b5ff105dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:41.396923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6330f-6fed-4dd2-92d9-e80b5ff105dc service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:41.397205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6330f-6fed-4dd2-92d9-e80b5ff105dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:41.397481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6330f-6fed-4dd2-92d9-e80b5ff105dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:41.406293 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6330f-6fed-4dd2-92d9-e80b5ff105dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:41.407761 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6330f-6fed-4dd2-92d9-e80b5ff105dc service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:41.417658 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1302/2634] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:41.837411 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96eb25-f1e3-4f62-bb81-9582f608e20b None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:41.838916 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-aa96eb25-f1e3-4f62-bb81-9582f608e20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:41.850940 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96eb25-f1e3-4f62-bb81-9582f608e20b None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:41.855611 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35889151-6d69-43ea-901c-def540f8d8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:41.856248 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35889151-6d69-43ea-901c-def540f8d8c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:41.856530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35889151-6d69-43ea-901c-def540f8d8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:41.856805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35889151-6d69-43ea-901c-def540f8d8c8 admin admin] PATH_INFO: `/v3/projects/2b8fc3d07b5a4d1d9a14fbfeb032a52d` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:41.858029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35889151-6d69-43ea-901c-def540f8d8c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b8fc3d07b5a4d1d9a14fbfeb032a52d)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:41.859966 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35889151-6d69-43ea-901c-def540f8d8c8 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:41.983893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-35889151-6d69-43ea-901c-def540f8d8c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b8fc3d07b5a4d1d9a14fbfeb032a52d', 'request_id': 'req-35889151-6d69-43ea-901c-def540f8d8c8'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:41.990828 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-35889151-6d69-43ea-901c-def540f8d8c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b8fc3d07b5a4d1d9a14fbfeb032a52d', 'request_id': 'req-35889151-6d69-43ea-901c-def540f8d8c8'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:42.083641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-35889151-6d69-43ea-901c-def540f8d8c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b8fc3d07b5a4d1d9a14fbfeb032a52d', 'request_id': 'req-35889151-6d69-43ea-901c-def540f8d8c8'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:42.094489 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-35889151-6d69-43ea-901c-def540f8d8c8 admin admin] The token cache is being invalidate because project 2b8fc3d07b5a4d1d9a14fbfeb032a52d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:15:42.094778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-35889151-6d69-43ea-901c-def540f8d8c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:42.095509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-35889151-6d69-43ea-901c-def540f8d8c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:42.097285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1333/2635] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:15:41 2022] DELETE /identity/v3/projects/2b8fc3d07b5a4d1d9a14fbfeb032a52d => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:42.101258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6330f-6fed-4dd2-92d9-e80b5ff105dc service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:42.110691 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6330f-6fed-4dd2-92d9-e80b5ff105dc service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:42.164995 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f5d93-287b-49dd-a3f2-8b36c5be98a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:42.165613 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f5d93-287b-49dd-a3f2-8b36c5be98a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:42.165892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f5d93-287b-49dd-a3f2-8b36c5be98a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:42.166164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f5d93-287b-49dd-a3f2-8b36c5be98a4 admin admin] PATH_INFO: `/v3/users/524de55e6da94d2799da3d2d883d8e24` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:42.167294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f5d93-287b-49dd-a3f2-8b36c5be98a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=524de55e6da94d2799da3d2d883d8e24)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:42.169175 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f5d93-287b-49dd-a3f2-8b36c5be98a4 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:42.336675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3c3f5d93-287b-49dd-a3f2-8b36c5be98a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '524de55e6da94d2799da3d2d883d8e24'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:42.420686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3c3f5d93-287b-49dd-a3f2-8b36c5be98a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '524de55e6da94d2799da3d2d883d8e24'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:42.421336 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3c3f5d93-287b-49dd-a3f2-8b36c5be98a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '524de55e6da94d2799da3d2d883d8e24'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:42.434509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3c3f5d93-287b-49dd-a3f2-8b36c5be98a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '524de55e6da94d2799da3d2d883d8e24'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:42.441343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3c3f5d93-287b-49dd-a3f2-8b36c5be98a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '524de55e6da94d2799da3d2d883d8e24'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:42.447212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1303/2636] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:15:42 2022] DELETE /identity/v3/users/524de55e6da94d2799da3d2d883d8e24 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:42.845837 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35889151-6d69-43ea-901c-def540f8d8c8 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:42.847434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-35889151-6d69-43ea-901c-def540f8d8c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:42.860673 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35889151-6d69-43ea-901c-def540f8d8c8 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:42.909059 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:42.918418 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae1f0c-edb4-43af-afc7-ae1998df5c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:42.919081 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae1f0c-edb4-43af-afc7-ae1998df5c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:42.919369 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae1f0c-edb4-43af-afc7-ae1998df5c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:42.919642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae1f0c-edb4-43af-afc7-ae1998df5c6e admin admin] PATH_INFO: `/v3/projects/ad5aa8b4b6fd495d82f12278229589dd` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:42.920742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae1f0c-edb4-43af-afc7-ae1998df5c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad5aa8b4b6fd495d82f12278229589dd)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:42.922665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae1f0c-edb4-43af-afc7-ae1998df5c6e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:43.024328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-32ae1f0c-edb4-43af-afc7-ae1998df5c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad5aa8b4b6fd495d82f12278229589dd', 'request_id': 'req-32ae1f0c-edb4-43af-afc7-ae1998df5c6e'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:43.031823 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-32ae1f0c-edb4-43af-afc7-ae1998df5c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad5aa8b4b6fd495d82f12278229589dd', 'request_id': 'req-32ae1f0c-edb4-43af-afc7-ae1998df5c6e'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:43.101270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-32ae1f0c-edb4-43af-afc7-ae1998df5c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad5aa8b4b6fd495d82f12278229589dd', 'request_id': 'req-32ae1f0c-edb4-43af-afc7-ae1998df5c6e'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:43.112765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-32ae1f0c-edb4-43af-afc7-ae1998df5c6e admin admin] The token cache is being invalidate because project ad5aa8b4b6fd495d82f12278229589dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:15:43.112962 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-32ae1f0c-edb4-43af-afc7-ae1998df5c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:43.113610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-32ae1f0c-edb4-43af-afc7-ae1998df5c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:43.115128 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1334/2637] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:15:42 2022] DELETE /identity/v3/projects/ad5aa8b4b6fd495d82f12278229589dd => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:43.127657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b57d1-ec9b-44f7-802e-52a9abf0e7b9 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:43.128159 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b57d1-ec9b-44f7-802e-52a9abf0e7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:43.128276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b57d1-ec9b-44f7-802e-52a9abf0e7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:43.148051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-9f0b57d1-ec9b-44f7-802e-52a9abf0e7b9 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:43.149246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9f0b57d1-ec9b-44f7-802e-52a9abf0e7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:43.158537 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:43.208676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1304/2638] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:43.216731 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ccc86-5c2b-47c0-84e4-a217e594cf0f None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:43.217063 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ccc86-5c2b-47c0-84e4-a217e594cf0f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:43.217448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ccc86-5c2b-47c0-84e4-a217e594cf0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:43.238502 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-5c9ccc86-5c2b-47c0-84e4-a217e594cf0f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:43.239762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5c9ccc86-5c2b-47c0-84e4-a217e594cf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:43.247800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1335/2639] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:43.252343 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b57d1-ec9b-44f7-802e-52a9abf0e7b9 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:43.262444 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b57d1-ec9b-44f7-802e-52a9abf0e7b9 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:43.267062 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c2c5b-dae5-4e7e-9e6e-11b99415e527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:43.267668 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c2c5b-dae5-4e7e-9e6e-11b99415e527 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:43.267947 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c2c5b-dae5-4e7e-9e6e-11b99415e527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:43.268222 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c2c5b-dae5-4e7e-9e6e-11b99415e527 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:43.268940 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c2c5b-dae5-4e7e-9e6e-11b99415e527 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:15:43.269598 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c2c5b-dae5-4e7e-9e6e-11b99415e527 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:43.271863 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c2c5b-dae5-4e7e-9e6e-11b99415e527 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:43.283683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1305/2640] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 15:15:43.288545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ccc86-5c2b-47c0-84e4-a217e594cf0f None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:43.297680 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ccc86-5c2b-47c0-84e4-a217e594cf0f None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:43.302910 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c43a34-ea3a-4377-a78d-a189648ba926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:43.303511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c43a34-ea3a-4377-a78d-a189648ba926 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:43.303789 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c43a34-ea3a-4377-a78d-a189648ba926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:43.304055 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c43a34-ea3a-4377-a78d-a189648ba926 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:43.304670 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c43a34-ea3a-4377-a78d-a189648ba926 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:43.306570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c43a34-ea3a-4377-a78d-a189648ba926 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:43.414298 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1336/2641] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:15:43 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:15:43.418337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c2c5b-dae5-4e7e-9e6e-11b99415e527 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:43.428967 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c2c5b-dae5-4e7e-9e6e-11b99415e527 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:43.482270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9378f1-36ad-43eb-8122-98e2a200c088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:43.482892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9378f1-36ad-43eb-8122-98e2a200c088 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:43.483171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9378f1-36ad-43eb-8122-98e2a200c088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:43.483442 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9378f1-36ad-43eb-8122-98e2a200c088 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:43.484060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9378f1-36ad-43eb-8122-98e2a200c088 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:43.485955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9378f1-36ad-43eb-8122-98e2a200c088 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:43.560607 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1306/2642] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:15:43 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:15:43.564711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c43a34-ea3a-4377-a78d-a189648ba926 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:43.575394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c43a34-ea3a-4377-a78d-a189648ba926 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:43.579906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66366d1d-423b-4e5c-83a9-e18e760e763d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:43.580498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66366d1d-423b-4e5c-83a9-e18e760e763d admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:43.580780 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66366d1d-423b-4e5c-83a9-e18e760e763d admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:43.581509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66366d1d-423b-4e5c-83a9-e18e760e763d admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:43.581726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66366d1d-423b-4e5c-83a9-e18e760e763d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:43.583284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66366d1d-423b-4e5c-83a9-e18e760e763d admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:43.593365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1337/2643] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:15:43.597134 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9378f1-36ad-43eb-8122-98e2a200c088 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:43.606842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9378f1-36ad-43eb-8122-98e2a200c088 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:43.611329 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5056c-f586-444b-8182-5166c77eb2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:43.612387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5056c-f586-444b-8182-5166c77eb2ef admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:43.612676 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5056c-f586-444b-8182-5166c77eb2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:43.612940 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5056c-f586-444b-8182-5166c77eb2ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:43.613565 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5056c-f586-444b-8182-5166c77eb2ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:43.615164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5056c-f586-444b-8182-5166c77eb2ef admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:43.627399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1307/2644] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 15:15:43.629790 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66366d1d-423b-4e5c-83a9-e18e760e763d admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:43.639762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66366d1d-423b-4e5c-83a9-e18e760e763d admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:43.644295 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190c2df-1293-4079-a3e0-9aeb4763b8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:43.644933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190c2df-1293-4079-a3e0-9aeb4763b8e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:43.645212 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190c2df-1293-4079-a3e0-9aeb4763b8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:43.645487 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190c2df-1293-4079-a3e0-9aeb4763b8e4 admin admin] PATH_INFO: `/v3/projects/4c4eff845c63494d8fb62d84d0ddc1e4/users/03b4a9747d8e4575acf751ada954e5a6/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:43.659530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190c2df-1293-4079-a3e0-9aeb4763b8e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c4eff845c63494d8fb62d84d0ddc1e4, user_id=03b4a9747d8e4575acf751ada954e5a6, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:43.662124 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190c2df-1293-4079-a3e0-9aeb4763b8e4 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:43.760855 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1338/2645] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:15:43 2022] PUT /identity/v3/projects/4c4eff845c63494d8fb62d84d0ddc1e4/users/03b4a9747d8e4575acf751ada954e5a6/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:43.771902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efad1c9-e8d5-4638-ace7-e6c10457af11 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:43.772195 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efad1c9-e8d5-4638-ace7-e6c10457af11 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:43.772472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efad1c9-e8d5-4638-ace7-e6c10457af11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:43.800771 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-5efad1c9-e8d5-4638-ace7-e6c10457af11 None None] MFA Rules not processed for user `03b4a9747d8e4575acf751ada954e5a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:43.801984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-5efad1c9-e8d5-4638-ace7-e6c10457af11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:43.863818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1308/2646] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:43.868430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190c2df-1293-4079-a3e0-9aeb4763b8e4 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:43.928174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190c2df-1293-4079-a3e0-9aeb4763b8e4 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:43.932781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398d2ad-2825-4886-8098-f1b55a60cf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:43.933379 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398d2ad-2825-4886-8098-f1b55a60cf01 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:43.933660 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398d2ad-2825-4886-8098-f1b55a60cf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:43.933932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398d2ad-2825-4886-8098-f1b55a60cf01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:43.934549 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398d2ad-2825-4886-8098-f1b55a60cf01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:43.936479 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398d2ad-2825-4886-8098-f1b55a60cf01 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:44.013506 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1339/2647] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:15:43 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:15:44.017649 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efad1c9-e8d5-4638-ace7-e6c10457af11 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:44.028453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efad1c9-e8d5-4638-ace7-e6c10457af11 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:44.036684 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:44.083681 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fad46-25b7-4b45-b21a-c70a7f13e96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:44.084285 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fad46-25b7-4b45-b21a-c70a7f13e96a admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:44.084563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fad46-25b7-4b45-b21a-c70a7f13e96a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:44.084834 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fad46-25b7-4b45-b21a-c70a7f13e96a admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:44.085462 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fad46-25b7-4b45-b21a-c70a7f13e96a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:44.087401 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fad46-25b7-4b45-b21a-c70a7f13e96a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:44.221258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1309/2648] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:15:44 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:15:44.225789 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398d2ad-2825-4886-8098-f1b55a60cf01 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:44.231277 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:44.236527 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398d2ad-2825-4886-8098-f1b55a60cf01 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:44.241050 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda21724-81f0-4c05-af39-15a1848525e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:44.241641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda21724-81f0-4c05-af39-15a1848525e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:44.241914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda21724-81f0-4c05-af39-15a1848525e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:44.242181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda21724-81f0-4c05-af39-15a1848525e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:44.242834 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda21724-81f0-4c05-af39-15a1848525e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:44.244407 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda21724-81f0-4c05-af39-15a1848525e1 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:44.254589 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1340/2649] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 15:15:44.258736 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fad46-25b7-4b45-b21a-c70a7f13e96a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:44.269361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fad46-25b7-4b45-b21a-c70a7f13e96a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:44.273775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c986e-c217-4ad2-8502-b5ce0c978004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:44.274358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c986e-c217-4ad2-8502-b5ce0c978004 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:44.274637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c986e-c217-4ad2-8502-b5ce0c978004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:44.274968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c986e-c217-4ad2-8502-b5ce0c978004 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:44.275604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c986e-c217-4ad2-8502-b5ce0c978004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:44.277220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c986e-c217-4ad2-8502-b5ce0c978004 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:44.288414 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1310/2650] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:15:44.292515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda21724-81f0-4c05-af39-15a1848525e1 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:44.302602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda21724-81f0-4c05-af39-15a1848525e1 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:44.307240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9debd-b669-4174-8d75-ceedf1b8213d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:44.307891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9debd-b669-4174-8d75-ceedf1b8213d admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:44.308225 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9debd-b669-4174-8d75-ceedf1b8213d admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:44.308450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9debd-b669-4174-8d75-ceedf1b8213d admin admin] PATH_INFO: `/v3/projects/fa1f999f80464d0398e978121d0fac2b/users/4fc265965ef2423f9f65ec0f345b360f/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:44.323842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9debd-b669-4174-8d75-ceedf1b8213d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa1f999f80464d0398e978121d0fac2b, user_id=4fc265965ef2423f9f65ec0f345b360f, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:44.326475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9debd-b669-4174-8d75-ceedf1b8213d admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:44.435115 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1341/2651] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:15:44 2022] PUT /identity/v3/projects/fa1f999f80464d0398e978121d0fac2b/users/4fc265965ef2423f9f65ec0f345b360f/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:15:44.503912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bb134-baf2-42c6-a878-8fd40b4c602a None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:44.504200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bb134-baf2-42c6-a878-8fd40b4c602a None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:44.504479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bb134-baf2-42c6-a878-8fd40b4c602a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:44.533153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-164bb134-baf2-42c6-a878-8fd40b4c602a None None] MFA Rules not processed for user `4fc265965ef2423f9f65ec0f345b360f`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:44.534326 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-164bb134-baf2-42c6-a878-8fd40b4c602a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:44.596175 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1311/2652] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 15:15:44.600652 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9debd-b669-4174-8d75-ceedf1b8213d admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:44.660109 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9debd-b669-4174-8d75-ceedf1b8213d admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:44.665071 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eca1e-18b8-4d9d-a8f0-0c11d904369e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:44.669254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eca1e-18b8-4d9d-a8f0-0c11d904369e admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:44.669254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eca1e-18b8-4d9d-a8f0-0c11d904369e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:44.669254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eca1e-18b8-4d9d-a8f0-0c11d904369e admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:44.669254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eca1e-18b8-4d9d-a8f0-0c11d904369e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:44.669254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eca1e-18b8-4d9d-a8f0-0c11d904369e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:44.745184 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1342/2653] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:15:44 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:15:44.749530 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164bb134-baf2-42c6-a878-8fd40b4c602a None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:44.760825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164bb134-baf2-42c6-a878-8fd40b4c602a None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:44.816564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77280a67-fa0c-40b1-b477-73543eeec696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:44.817124 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77280a67-fa0c-40b1-b477-73543eeec696 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:44.817402 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77280a67-fa0c-40b1-b477-73543eeec696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:44.817670 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77280a67-fa0c-40b1-b477-73543eeec696 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:44.818296 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77280a67-fa0c-40b1-b477-73543eeec696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:44.820215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77280a67-fa0c-40b1-b477-73543eeec696 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:44.903479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1312/2654] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:15:44 2022] POST /identity/v3/users => generated 299 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:15:44.907500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eca1e-18b8-4d9d-a8f0-0c11d904369e admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:44.917618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eca1e-18b8-4d9d-a8f0-0c11d904369e admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:44.922240 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c44635-c3d1-4625-a3c8-8fb8ca5bb3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:44.922860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c44635-c3d1-4625-a3c8-8fb8ca5bb3be admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:44.923142 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c44635-c3d1-4625-a3c8-8fb8ca5bb3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:44.923413 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c44635-c3d1-4625-a3c8-8fb8ca5bb3be admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:44.924043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c44635-c3d1-4625-a3c8-8fb8ca5bb3be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:44.926131 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c44635-c3d1-4625-a3c8-8fb8ca5bb3be admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:44.936433 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1343/2655] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:15:44.940889 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77280a67-fa0c-40b1-b477-73543eeec696 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:44.950530 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77280a67-fa0c-40b1-b477-73543eeec696 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:44.955123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75c098-aa7e-46d4-b83d-426d76442ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:44.955800 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75c098-aa7e-46d4-b83d-426d76442ba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:44.956078 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75c098-aa7e-46d4-b83d-426d76442ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:44.956351 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75c098-aa7e-46d4-b83d-426d76442ba6 admin admin] PATH_INFO: `/v3/projects/d7f729529c654b54997ece9d3ae8b7eb/users/ec48be1f6c7e412aab538298c78cb089/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:44.959311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:44.970742 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75c098-aa7e-46d4-b83d-426d76442ba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7f729529c654b54997ece9d3ae8b7eb, user_id=ec48be1f6c7e412aab538298c78cb089, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:44.973463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75c098-aa7e-46d4-b83d-426d76442ba6 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:45.058316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1313/2656] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:15:44 2022] PUT /identity/v3/projects/d7f729529c654b54997ece9d3ae8b7eb/users/ec48be1f6c7e412aab538298c78cb089/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:15:45.062323 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c44635-c3d1-4625-a3c8-8fb8ca5bb3be admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:45.073562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c44635-c3d1-4625-a3c8-8fb8ca5bb3be admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:45.128765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333bd3d-b777-4af6-8c40-1da0441a9aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:45.129362 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333bd3d-b777-4af6-8c40-1da0441a9aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:45.129643 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333bd3d-b777-4af6-8c40-1da0441a9aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:45.129915 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333bd3d-b777-4af6-8c40-1da0441a9aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:45.130545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333bd3d-b777-4af6-8c40-1da0441a9aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:45.132404 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333bd3d-b777-4af6-8c40-1da0441a9aa4 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:45.141758 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1344/2657] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:15:45.145975 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75c098-aa7e-46d4-b83d-426d76442ba6 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:45.156228 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75c098-aa7e-46d4-b83d-426d76442ba6 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:45.160758 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6322a25-1596-45fa-81b5-a722fc358202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:45.161399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6322a25-1596-45fa-81b5-a722fc358202 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:45.161680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6322a25-1596-45fa-81b5-a722fc358202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:45.161955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6322a25-1596-45fa-81b5-a722fc358202 admin admin] PATH_INFO: `/v3/projects/d7f729529c654b54997ece9d3ae8b7eb/users/ec48be1f6c7e412aab538298c78cb089/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:45.163929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6322a25-1596-45fa-81b5-a722fc358202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7f729529c654b54997ece9d3ae8b7eb, user_id=ec48be1f6c7e412aab538298c78cb089, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:45.166595 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6322a25-1596-45fa-81b5-a722fc358202 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:45.177099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1314/2658] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:15:45 2022] PUT /identity/v3/projects/d7f729529c654b54997ece9d3ae8b7eb/users/ec48be1f6c7e412aab538298c78cb089/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:45.187265 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ae48e-c7bc-4e56-9811-3e39039391bc None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:45.187551 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ae48e-c7bc-4e56-9811-3e39039391bc None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:45.187826 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ae48e-c7bc-4e56-9811-3e39039391bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:45.215283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-4f7ae48e-c7bc-4e56-9811-3e39039391bc None None] MFA Rules not processed for user `ec48be1f6c7e412aab538298c78cb089`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:45.216491 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-4f7ae48e-c7bc-4e56-9811-3e39039391bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:45.242172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:45.283522 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1345/2659] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 15:15:45.295662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6322a25-1596-45fa-81b5-a722fc358202 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:45.297384 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c6322a25-1596-45fa-81b5-a722fc358202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:45.355715 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6322a25-1596-45fa-81b5-a722fc358202 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:45.359384 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f09ad-cacd-451b-baca-682a82440bb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:45.359990 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f09ad-cacd-451b-baca-682a82440bb8 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:45.360270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f09ad-cacd-451b-baca-682a82440bb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:45.360544 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f09ad-cacd-451b-baca-682a82440bb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:45.418594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f09ad-cacd-451b-baca-682a82440bb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:45.420060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f09ad-cacd-451b-baca-682a82440bb8 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:45.429326 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1315/2660] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:47.398177 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ae48e-c7bc-4e56-9811-3e39039391bc None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:47.407701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ae48e-c7bc-4e56-9811-3e39039391bc None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:47.411848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4618a9-8c9b-4dc6-9578-ae9c18f88e31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:47.412441 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4618a9-8c9b-4dc6-9578-ae9c18f88e31 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:47.412723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4618a9-8c9b-4dc6-9578-ae9c18f88e31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:47.412993 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4618a9-8c9b-4dc6-9578-ae9c18f88e31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:47.468548 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4618a9-8c9b-4dc6-9578-ae9c18f88e31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:47.469967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4618a9-8c9b-4dc6-9578-ae9c18f88e31 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:47.479459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1346/2661] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:48.848987 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f09ad-cacd-451b-baca-682a82440bb8 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:48.858145 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f09ad-cacd-451b-baca-682a82440bb8 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:48.861702 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a746189-903f-4f3a-ad7a-dc94ba3cd9ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:48.862293 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a746189-903f-4f3a-ad7a-dc94ba3cd9ef service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:48.862569 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a746189-903f-4f3a-ad7a-dc94ba3cd9ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:48.862870 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a746189-903f-4f3a-ad7a-dc94ba3cd9ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:48.871776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a746189-903f-4f3a-ad7a-dc94ba3cd9ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:48.873674 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a746189-903f-4f3a-ad7a-dc94ba3cd9ef service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:48.883144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1316/2662] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:52.229564 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4618a9-8c9b-4dc6-9578-ae9c18f88e31 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:52.231133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ab4618a9-8c9b-4dc6-9578-ae9c18f88e31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:52.291350 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4618a9-8c9b-4dc6-9578-ae9c18f88e31 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:52.295134 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196257b3-b469-4090-9e7b-5c078bfc9d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:52.295739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196257b3-b469-4090-9e7b-5c078bfc9d11 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:52.296020 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196257b3-b469-4090-9e7b-5c078bfc9d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:52.296292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196257b3-b469-4090-9e7b-5c078bfc9d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:52.297993 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-196257b3-b469-4090-9e7b-5c078bfc9d11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:52.308040 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196257b3-b469-4090-9e7b-5c078bfc9d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:52.309452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196257b3-b469-4090-9e7b-5c078bfc9d11 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:52.317484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1347/2663] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:53.473055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a746189-903f-4f3a-ad7a-dc94ba3cd9ef service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:53.533992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a746189-903f-4f3a-ad7a-dc94ba3cd9ef service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:53.538580 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bacd7f-93bc-4974-ab43-8ca86b3522e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:53.539246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bacd7f-93bc-4974-ab43-8ca86b3522e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:53.539529 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bacd7f-93bc-4974-ab43-8ca86b3522e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:53.539807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bacd7f-93bc-4974-ab43-8ca86b3522e1 admin admin] PATH_INFO: `/v3/users/03b4a9747d8e4575acf751ada954e5a6` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:53.540931 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bacd7f-93bc-4974-ab43-8ca86b3522e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03b4a9747d8e4575acf751ada954e5a6)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:53.542901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bacd7f-93bc-4974-ab43-8ca86b3522e1 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:53.556531 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:53.708945 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e2bacd7f-93bc-4974-ab43-8ca86b3522e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03b4a9747d8e4575acf751ada954e5a6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:53.751912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e2bacd7f-93bc-4974-ab43-8ca86b3522e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03b4a9747d8e4575acf751ada954e5a6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:53.752541 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e2bacd7f-93bc-4974-ab43-8ca86b3522e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03b4a9747d8e4575acf751ada954e5a6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:53.765175 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e2bacd7f-93bc-4974-ab43-8ca86b3522e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03b4a9747d8e4575acf751ada954e5a6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:53.772572 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e2bacd7f-93bc-4974-ab43-8ca86b3522e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03b4a9747d8e4575acf751ada954e5a6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:53.778412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1317/2664] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:15:53 2022] DELETE /identity/v3/users/03b4a9747d8e4575acf751ada954e5a6 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:53.783221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ea3e6-d054-46e0-869b-88badd408555 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:53.783623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ea3e6-d054-46e0-869b-88badd408555 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:53.783766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ea3e6-d054-46e0-869b-88badd408555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:53.802222 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-112ea3e6-d054-46e0-869b-88badd408555 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:53.803996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-112ea3e6-d054-46e0-869b-88badd408555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:53.862440 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1348/2665] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:53.872480 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bacd7f-93bc-4974-ab43-8ca86b3522e1 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:53.873884 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e2bacd7f-93bc-4974-ab43-8ca86b3522e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:53.929809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bacd7f-93bc-4974-ab43-8ca86b3522e1 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:53.933411 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3c8b2-0225-4813-99f7-13a598bf4bbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:53.934010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3c8b2-0225-4813-99f7-13a598bf4bbc service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:53.934292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3c8b2-0225-4813-99f7-13a598bf4bbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:53.934567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3c8b2-0225-4813-99f7-13a598bf4bbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:53.943445 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3c8b2-0225-4813-99f7-13a598bf4bbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:53.945463 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3c8b2-0225-4813-99f7-13a598bf4bbc service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:53.954714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1318/2666] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:15:54.357132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ea3e6-d054-46e0-869b-88badd408555 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:54.358630 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-112ea3e6-d054-46e0-869b-88badd408555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:54.368656 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:54.373007 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ea3e6-d054-46e0-869b-88badd408555 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:54.377920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6309677-36a2-45f0-8c1c-829cd422be88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:54.382939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6309677-36a2-45f0-8c1c-829cd422be88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:54.382939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6309677-36a2-45f0-8c1c-829cd422be88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:54.382939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6309677-36a2-45f0-8c1c-829cd422be88 admin admin] PATH_INFO: `/v3/projects/4c4eff845c63494d8fb62d84d0ddc1e4` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:54.382939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6309677-36a2-45f0-8c1c-829cd422be88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c4eff845c63494d8fb62d84d0ddc1e4)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:54.382939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6309677-36a2-45f0-8c1c-829cd422be88 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:54.504003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a6309677-36a2-45f0-8c1c-829cd422be88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c4eff845c63494d8fb62d84d0ddc1e4', 'request_id': 'req-a6309677-36a2-45f0-8c1c-829cd422be88'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:54.510265 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a6309677-36a2-45f0-8c1c-829cd422be88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c4eff845c63494d8fb62d84d0ddc1e4', 'request_id': 'req-a6309677-36a2-45f0-8c1c-829cd422be88'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:54.577985 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a6309677-36a2-45f0-8c1c-829cd422be88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c4eff845c63494d8fb62d84d0ddc1e4', 'request_id': 'req-a6309677-36a2-45f0-8c1c-829cd422be88'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:54.589363 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a6309677-36a2-45f0-8c1c-829cd422be88 admin admin] The token cache is being invalidate because project 4c4eff845c63494d8fb62d84d0ddc1e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:15:54.589651 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a6309677-36a2-45f0-8c1c-829cd422be88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:54.590262 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a6309677-36a2-45f0-8c1c-829cd422be88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:54.591797 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1349/2667] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:15:54 2022] DELETE /identity/v3/projects/4c4eff845c63494d8fb62d84d0ddc1e4 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:54.595933 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3c8b2-0225-4813-99f7-13a598bf4bbc service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:54.606174 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3c8b2-0225-4813-99f7-13a598bf4bbc service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:54.659623 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b4fca-d505-4abe-aaa4-8ac408f98810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:54.660275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b4fca-d505-4abe-aaa4-8ac408f98810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:54.663694 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b4fca-d505-4abe-aaa4-8ac408f98810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:54.663694 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b4fca-d505-4abe-aaa4-8ac408f98810 admin admin] PATH_INFO: `/v3/users/4fc265965ef2423f9f65ec0f345b360f` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:54.663694 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b4fca-d505-4abe-aaa4-8ac408f98810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fc265965ef2423f9f65ec0f345b360f)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:54.664506 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b4fca-d505-4abe-aaa4-8ac408f98810 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:54.825820 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:54.833873 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1f4b4fca-d505-4abe-aaa4-8ac408f98810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fc265965ef2423f9f65ec0f345b360f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:54.906622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1f4b4fca-d505-4abe-aaa4-8ac408f98810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fc265965ef2423f9f65ec0f345b360f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:54.907280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1f4b4fca-d505-4abe-aaa4-8ac408f98810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fc265965ef2423f9f65ec0f345b360f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:54.920068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1f4b4fca-d505-4abe-aaa4-8ac408f98810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fc265965ef2423f9f65ec0f345b360f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:54.926896 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1f4b4fca-d505-4abe-aaa4-8ac408f98810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fc265965ef2423f9f65ec0f345b360f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:54.932919 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1319/2668] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:15:54 2022] DELETE /identity/v3/users/4fc265965ef2423f9f65ec0f345b360f => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:55.368481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6309677-36a2-45f0-8c1c-829cd422be88 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:55.368481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a6309677-36a2-45f0-8c1c-829cd422be88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:55.381918 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6309677-36a2-45f0-8c1c-829cd422be88 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:55.439603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:55.440236 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:55.440516 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:55.440790 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a admin admin] PATH_INFO: `/v3/projects/fa1f999f80464d0398e978121d0fac2b` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:55.441902 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa1f999f80464d0398e978121d0fac2b)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:55.443876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:55.563371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa1f999f80464d0398e978121d0fac2b', 'request_id': 'req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:55.569906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa1f999f80464d0398e978121d0fac2b', 'request_id': 'req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:55.638009 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa1f999f80464d0398e978121d0fac2b', 'request_id': 'req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:55.648824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a admin admin] The token cache is being invalidate because project fa1f999f80464d0398e978121d0fac2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:15:55.649109 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:55.649732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:55.651241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1350/2669] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:15:55 2022] DELETE /identity/v3/projects/fa1f999f80464d0398e978121d0fac2b => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:55.655154 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b4fca-d505-4abe-aaa4-8ac408f98810 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:55.656685 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-1f4b4fca-d505-4abe-aaa4-8ac408f98810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:55.670005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b4fca-d505-4abe-aaa4-8ac408f98810 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:55.724164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a11112-2bf5-4f05-ab9d-b400c93f40ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:55.724784 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a11112-2bf5-4f05-ab9d-b400c93f40ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:55.725106 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a11112-2bf5-4f05-ab9d-b400c93f40ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:55.725375 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a11112-2bf5-4f05-ab9d-b400c93f40ef admin admin] PATH_INFO: `/v3/users/ec48be1f6c7e412aab538298c78cb089` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:55.726459 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a11112-2bf5-4f05-ab9d-b400c93f40ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec48be1f6c7e412aab538298c78cb089)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:55.728388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a11112-2bf5-4f05-ab9d-b400c93f40ef admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:55.914730 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-25a11112-2bf5-4f05-ab9d-b400c93f40ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec48be1f6c7e412aab538298c78cb089'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:55.986410 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-25a11112-2bf5-4f05-ab9d-b400c93f40ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec48be1f6c7e412aab538298c78cb089'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:55.987083 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-25a11112-2bf5-4f05-ab9d-b400c93f40ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec48be1f6c7e412aab538298c78cb089'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:55.999407 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-25a11112-2bf5-4f05-ab9d-b400c93f40ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec48be1f6c7e412aab538298c78cb089'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:56.006681 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-25a11112-2bf5-4f05-ab9d-b400c93f40ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec48be1f6c7e412aab538298c78cb089'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:56.012815 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1320/2670] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:15:55 2022] DELETE /identity/v3/users/ec48be1f6c7e412aab538298c78cb089 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:56.417188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:56.418641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:56.433369 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6ae7b-861e-4a6f-a329-2c3c04cc973a admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:56.447497 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:56.491316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243a97a-c544-49f3-936e-c011152c9035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:56.491953 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243a97a-c544-49f3-936e-c011152c9035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:56.492237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243a97a-c544-49f3-936e-c011152c9035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:56.492517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243a97a-c544-49f3-936e-c011152c9035 admin admin] PATH_INFO: `/v3/projects/d7f729529c654b54997ece9d3ae8b7eb` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:56.493667 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243a97a-c544-49f3-936e-c011152c9035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7f729529c654b54997ece9d3ae8b7eb)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:56.495604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243a97a-c544-49f3-936e-c011152c9035 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:56.638318 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-9243a97a-c544-49f3-936e-c011152c9035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7f729529c654b54997ece9d3ae8b7eb', 'request_id': 'req-9243a97a-c544-49f3-936e-c011152c9035'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:56.644791 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-9243a97a-c544-49f3-936e-c011152c9035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7f729529c654b54997ece9d3ae8b7eb', 'request_id': 'req-9243a97a-c544-49f3-936e-c011152c9035'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:56.708981 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-9243a97a-c544-49f3-936e-c011152c9035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7f729529c654b54997ece9d3ae8b7eb', 'request_id': 'req-9243a97a-c544-49f3-936e-c011152c9035'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:56.719247 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-9243a97a-c544-49f3-936e-c011152c9035 admin admin] The token cache is being invalidate because project d7f729529c654b54997ece9d3ae8b7eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:15:56.723461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-9243a97a-c544-49f3-936e-c011152c9035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:56.723461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-9243a97a-c544-49f3-936e-c011152c9035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:15:56.723461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1351/2671] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:15:56 2022] DELETE /identity/v3/projects/d7f729529c654b54997ece9d3ae8b7eb => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:56.727147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ca181-c382-465a-8d7b-8922f7cd72af None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:56.727605 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ca181-c382-465a-8d7b-8922f7cd72af None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:56.727702 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ca181-c382-465a-8d7b-8922f7cd72af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:56.747597 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-ef2ca181-c382-465a-8d7b-8922f7cd72af None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:56.748809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-ef2ca181-c382-465a-8d7b-8922f7cd72af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:56.777574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:56.808043 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1321/2672] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:56.817065 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c08e5-c577-49e3-a2e3-7e8d9a3f038d None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:56.817876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c08e5-c577-49e3-a2e3-7e8d9a3f038d None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:56.817876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c08e5-c577-49e3-a2e3-7e8d9a3f038d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:56.837395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-172c08e5-c577-49e3-a2e3-7e8d9a3f038d None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:56.838596 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-172c08e5-c577-49e3-a2e3-7e8d9a3f038d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:56.846392 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1352/2673] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:15:56.851351 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ca181-c382-465a-8d7b-8922f7cd72af None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:56.860957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ca181-c382-465a-8d7b-8922f7cd72af None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:56.866023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9d527-d412-4fb2-8bd1-1aa06768ea91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:56.866652 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9d527-d412-4fb2-8bd1-1aa06768ea91 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:56.866963 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9d527-d412-4fb2-8bd1-1aa06768ea91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:56.867249 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9d527-d412-4fb2-8bd1-1aa06768ea91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:56.867892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb9d527-d412-4fb2-8bd1-1aa06768ea91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:15:56.868199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb9d527-d412-4fb2-8bd1-1aa06768ea91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:56.869806 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb9d527-d412-4fb2-8bd1-1aa06768ea91 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:56.881441 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1322/2674] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 15:15:56.886387 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c08e5-c577-49e3-a2e3-7e8d9a3f038d None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:56.895537 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c08e5-c577-49e3-a2e3-7e8d9a3f038d None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:56.900610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066b4ac-d10d-42a8-b4c5-461d3426c074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:56.901194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066b4ac-d10d-42a8-b4c5-461d3426c074 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:56.901472 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066b4ac-d10d-42a8-b4c5-461d3426c074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:56.901741 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066b4ac-d10d-42a8-b4c5-461d3426c074 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:56.902369 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066b4ac-d10d-42a8-b4c5-461d3426c074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:56.904272 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066b4ac-d10d-42a8-b4c5-461d3426c074 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:57.043036 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1353/2675] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:15:56 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:15:57.047175 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9d527-d412-4fb2-8bd1-1aa06768ea91 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:57.057201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9d527-d412-4fb2-8bd1-1aa06768ea91 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:57.114979 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e863c6e-12d4-4903-a1f9-c757f18bbec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:57.115589 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e863c6e-12d4-4903-a1f9-c757f18bbec4 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:57.115872 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e863c6e-12d4-4903-a1f9-c757f18bbec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:57.116146 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e863c6e-12d4-4903-a1f9-c757f18bbec4 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:57.116777 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e863c6e-12d4-4903-a1f9-c757f18bbec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:57.118763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e863c6e-12d4-4903-a1f9-c757f18bbec4 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:57.229756 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1323/2676] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:15:57 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:15:57.233687 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066b4ac-d10d-42a8-b4c5-461d3426c074 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:57.244073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066b4ac-d10d-42a8-b4c5-461d3426c074 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:57.248700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421396a-0fb6-41ec-af9e-14c6a1b15dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:57.249302 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421396a-0fb6-41ec-af9e-14c6a1b15dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:57.249583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421396a-0fb6-41ec-af9e-14c6a1b15dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:57.249856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421396a-0fb6-41ec-af9e-14c6a1b15dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:57.250495 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421396a-0fb6-41ec-af9e-14c6a1b15dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:57.252129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421396a-0fb6-41ec-af9e-14c6a1b15dc9 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:57.263269 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1354/2677] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:15:57.267224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e863c6e-12d4-4903-a1f9-c757f18bbec4 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:57.278004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e863c6e-12d4-4903-a1f9-c757f18bbec4 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:57.282694 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba98fd-b95c-45ab-a5ab-debf0c9bc5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:57.283320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba98fd-b95c-45ab-a5ab-debf0c9bc5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:57.283598 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba98fd-b95c-45ab-a5ab-debf0c9bc5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:57.283871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba98fd-b95c-45ab-a5ab-debf0c9bc5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:57.284505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba98fd-b95c-45ab-a5ab-debf0c9bc5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:57.286115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba98fd-b95c-45ab-a5ab-debf0c9bc5c0 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:57.297002 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1324/2678] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:15:57.301028 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421396a-0fb6-41ec-af9e-14c6a1b15dc9 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:57.310852 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421396a-0fb6-41ec-af9e-14c6a1b15dc9 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:57.315500 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffddeb5-3218-491a-a9f7-e6f2d58ec1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:57.316140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffddeb5-3218-491a-a9f7-e6f2d58ec1d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:57.316420 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffddeb5-3218-491a-a9f7-e6f2d58ec1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:57.316693 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffddeb5-3218-491a-a9f7-e6f2d58ec1d9 admin admin] PATH_INFO: `/v3/projects/95f6403fa1f64610836a9ff006afd8c3/users/5846273dd0fd4a6bb96e66dcb1e0e6b0/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:57.332376 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffddeb5-3218-491a-a9f7-e6f2d58ec1d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f6403fa1f64610836a9ff006afd8c3, user_id=5846273dd0fd4a6bb96e66dcb1e0e6b0, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:57.335660 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffddeb5-3218-491a-a9f7-e6f2d58ec1d9 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:57.418095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1355/2679] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:15:57 2022] PUT /identity/v3/projects/95f6403fa1f64610836a9ff006afd8c3/users/5846273dd0fd4a6bb96e66dcb1e0e6b0/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:15:57.429080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc389e13-1035-46e7-a721-d11e81f458a7 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:57.434167 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc389e13-1035-46e7-a721-d11e81f458a7 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:57.434167 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc389e13-1035-46e7-a721-d11e81f458a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:57.458221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-cc389e13-1035-46e7-a721-d11e81f458a7 None None] MFA Rules not processed for user `5846273dd0fd4a6bb96e66dcb1e0e6b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:15:57.459453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-cc389e13-1035-46e7-a721-d11e81f458a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:57.521869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1325/2680] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:15:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 15:15:57.532885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffddeb5-3218-491a-a9f7-e6f2d58ec1d9 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:15:57.534570 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5ffddeb5-3218-491a-a9f7-e6f2d58ec1d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:15:57.580674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:15:57.591597 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffddeb5-3218-491a-a9f7-e6f2d58ec1d9 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:15:57.595311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60bfb0-7112-46d3-9898-27feaa04b342 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:15:57.595910 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60bfb0-7112-46d3-9898-27feaa04b342 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:15:57.596193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60bfb0-7112-46d3-9898-27feaa04b342 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:15:57.596464 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60bfb0-7112-46d3-9898-27feaa04b342 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:15:57.605127 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60bfb0-7112-46d3-9898-27feaa04b342 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:15:57.606540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60bfb0-7112-46d3-9898-27feaa04b342 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:15:57.615726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1356/2681] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:15:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:16:00.613503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc389e13-1035-46e7-a721-d11e81f458a7 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:16:00.643297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:16:00.673930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc389e13-1035-46e7-a721-d11e81f458a7 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:16:00.678977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05218e3e-eef4-4fa1-8e47-c0e09d2559b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:16:00.679601 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05218e3e-eef4-4fa1-8e47-c0e09d2559b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:00.679883 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05218e3e-eef4-4fa1-8e47-c0e09d2559b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:00.680160 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05218e3e-eef4-4fa1-8e47-c0e09d2559b4 admin admin] PATH_INFO: `/v3/users/5846273dd0fd4a6bb96e66dcb1e0e6b0` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:00.681262 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05218e3e-eef4-4fa1-8e47-c0e09d2559b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5846273dd0fd4a6bb96e66dcb1e0e6b0)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:16:00.683213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05218e3e-eef4-4fa1-8e47-c0e09d2559b4 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:16:00.862533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-05218e3e-eef4-4fa1-8e47-c0e09d2559b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5846273dd0fd4a6bb96e66dcb1e0e6b0'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:16:00.928966 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-05218e3e-eef4-4fa1-8e47-c0e09d2559b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5846273dd0fd4a6bb96e66dcb1e0e6b0'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:16:00.929604 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-05218e3e-eef4-4fa1-8e47-c0e09d2559b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5846273dd0fd4a6bb96e66dcb1e0e6b0'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:16:00.941520 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-05218e3e-eef4-4fa1-8e47-c0e09d2559b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5846273dd0fd4a6bb96e66dcb1e0e6b0'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:16:00.948964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-05218e3e-eef4-4fa1-8e47-c0e09d2559b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5846273dd0fd4a6bb96e66dcb1e0e6b0'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:16:00.954749 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1326/2682] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:16:00 2022] DELETE /identity/v3/users/5846273dd0fd4a6bb96e66dcb1e0e6b0 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:16:00.959291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68d8a2-beba-48cc-9650-639e08dd0ad9 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:00.960383 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68d8a2-beba-48cc-9650-639e08dd0ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:00.960476 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68d8a2-beba-48cc-9650-639e08dd0ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:00.978919 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-cf68d8a2-beba-48cc-9650-639e08dd0ad9 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:16:00.980122 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-cf68d8a2-beba-48cc-9650-639e08dd0ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:16:01.036881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1357/2683] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 15:16:01.046483 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05218e3e-eef4-4fa1-8e47-c0e09d2559b4 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:16:01.047913 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-05218e3e-eef4-4fa1-8e47-c0e09d2559b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:16:01.103927 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05218e3e-eef4-4fa1-8e47-c0e09d2559b4 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:16:01.107612 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ba3de-c873-472d-99c3-50f17930ec6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:16:01.108219 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ba3de-c873-472d-99c3-50f17930ec6c service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:01.108496 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ba3de-c873-472d-99c3-50f17930ec6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:01.108774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ba3de-c873-472d-99c3-50f17930ec6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:01.117674 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ba3de-c873-472d-99c3-50f17930ec6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:16:01.119095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ba3de-c873-472d-99c3-50f17930ec6c service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:16:01.128383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1327/2684] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:16:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:16:01.562259 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68d8a2-beba-48cc-9650-639e08dd0ad9 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:16:01.564774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-cf68d8a2-beba-48cc-9650-639e08dd0ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:16:01.581160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68d8a2-beba-48cc-9650-639e08dd0ad9 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:16:01.586608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a15e1-c664-436e-ad06-66f9a1393d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:16:01.587896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a15e1-c664-436e-ad06-66f9a1393d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:01.587896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a15e1-c664-436e-ad06-66f9a1393d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:01.588784 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a15e1-c664-436e-ad06-66f9a1393d05 admin admin] PATH_INFO: `/v3/projects/95f6403fa1f64610836a9ff006afd8c3` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:01.589975 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97a15e1-c664-436e-ad06-66f9a1393d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95f6403fa1f64610836a9ff006afd8c3)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:16:01.592102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97a15e1-c664-436e-ad06-66f9a1393d05 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:16:01.716501 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b97a15e1-c664-436e-ad06-66f9a1393d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95f6403fa1f64610836a9ff006afd8c3', 'request_id': 'req-b97a15e1-c664-436e-ad06-66f9a1393d05'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:16:01.722850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b97a15e1-c664-436e-ad06-66f9a1393d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95f6403fa1f64610836a9ff006afd8c3', 'request_id': 'req-b97a15e1-c664-436e-ad06-66f9a1393d05'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:16:01.798345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b97a15e1-c664-436e-ad06-66f9a1393d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95f6403fa1f64610836a9ff006afd8c3', 'request_id': 'req-b97a15e1-c664-436e-ad06-66f9a1393d05'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:16:01.809933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b97a15e1-c664-436e-ad06-66f9a1393d05 admin admin] The token cache is being invalidate because project 95f6403fa1f64610836a9ff006afd8c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:16:01.810220 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b97a15e1-c664-436e-ad06-66f9a1393d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:16:01.810882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-b97a15e1-c664-436e-ad06-66f9a1393d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:16:01.812369 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1358/2685] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:16:01 2022] DELETE /identity/v3/projects/95f6403fa1f64610836a9ff006afd8c3 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:16:01.817246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d6cbe-a24f-4d47-ad73-fe6cde0b421f None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:01.817708 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d6cbe-a24f-4d47-ad73-fe6cde0b421f None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:01.817811 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d6cbe-a24f-4d47-ad73-fe6cde0b421f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:01.837123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-f39d6cbe-a24f-4d47-ad73-fe6cde0b421f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:16:01.838316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f39d6cbe-a24f-4d47-ad73-fe6cde0b421f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:16:01.884237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:16:01.898177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1328/2686] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:16:01.906995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd3281-b9a0-4db5-bb4f-58423596b056 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:01.907285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd3281-b9a0-4db5-bb4f-58423596b056 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:01.907562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd3281-b9a0-4db5-bb4f-58423596b056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:01.927509 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-c2cd3281-b9a0-4db5-bb4f-58423596b056 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:16:01.928718 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c2cd3281-b9a0-4db5-bb4f-58423596b056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:16:01.938157 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1359/2687] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:16:01.942411 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d6cbe-a24f-4d47-ad73-fe6cde0b421f None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:16:01.953653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d6cbe-a24f-4d47-ad73-fe6cde0b421f None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:16:01.958322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a677a1d-8bcb-4e1f-ac1a-c7fc7bd407a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:16:01.958959 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a677a1d-8bcb-4e1f-ac1a-c7fc7bd407a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:01.959240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a677a1d-8bcb-4e1f-ac1a-c7fc7bd407a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:01.959513 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a677a1d-8bcb-4e1f-ac1a-c7fc7bd407a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:01.960151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a677a1d-8bcb-4e1f-ac1a-c7fc7bd407a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:16:01.960497 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a677a1d-8bcb-4e1f-ac1a-c7fc7bd407a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:16:01.962088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a677a1d-8bcb-4e1f-ac1a-c7fc7bd407a3 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:16:01.975561 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1329/2688] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 15:16:01.983003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd3281-b9a0-4db5-bb4f-58423596b056 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:16:01.992844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd3281-b9a0-4db5-bb4f-58423596b056 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:16:01.997560 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb3ac9-8fb8-4eb4-9623-1a2b92fbb1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:16:01.998149 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb3ac9-8fb8-4eb4-9623-1a2b92fbb1e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:01.998428 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb3ac9-8fb8-4eb4-9623-1a2b92fbb1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:01.998699 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb3ac9-8fb8-4eb4-9623-1a2b92fbb1e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:01.999353 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb3ac9-8fb8-4eb4-9623-1a2b92fbb1e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:16:02.001218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb3ac9-8fb8-4eb4-9623-1a2b92fbb1e0 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:16:02.086060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1360/2689] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:16:01 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:16:02.090062 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a677a1d-8bcb-4e1f-ac1a-c7fc7bd407a3 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:16:02.100424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a677a1d-8bcb-4e1f-ac1a-c7fc7bd407a3 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:16:02.155898 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17dfbb-e1a1-4923-9edc-6bdc2db342b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:16:02.156473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17dfbb-e1a1-4923-9edc-6bdc2db342b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:02.156753 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17dfbb-e1a1-4923-9edc-6bdc2db342b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:02.157049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17dfbb-e1a1-4923-9edc-6bdc2db342b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:02.157737 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17dfbb-e1a1-4923-9edc-6bdc2db342b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:16:02.159717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17dfbb-e1a1-4923-9edc-6bdc2db342b4 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:16:02.275973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1330/2690] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:16:02 2022] POST /identity/v3/users => generated 283 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:16:02.280055 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb3ac9-8fb8-4eb4-9623-1a2b92fbb1e0 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:16:02.293852 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb3ac9-8fb8-4eb4-9623-1a2b92fbb1e0 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:16:02.301182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac23495-72ae-4b4f-8beb-3247179df577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:16:02.304617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac23495-72ae-4b4f-8beb-3247179df577 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:02.304918 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac23495-72ae-4b4f-8beb-3247179df577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:02.305196 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac23495-72ae-4b4f-8beb-3247179df577 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:02.305839 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac23495-72ae-4b4f-8beb-3247179df577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:16:02.310218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac23495-72ae-4b4f-8beb-3247179df577 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:16:02.320687 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1361/2691] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:16:02.324854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17dfbb-e1a1-4923-9edc-6bdc2db342b4 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:16:02.335913 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17dfbb-e1a1-4923-9edc-6bdc2db342b4 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:16:02.340682 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fd967-1966-44ca-b7d4-8b4a71b5f80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:16:02.341277 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fd967-1966-44ca-b7d4-8b4a71b5f80f admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:02.341557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fd967-1966-44ca-b7d4-8b4a71b5f80f admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:02.341830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fd967-1966-44ca-b7d4-8b4a71b5f80f admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:02.342466 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fd967-1966-44ca-b7d4-8b4a71b5f80f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:16:02.344106 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fd967-1966-44ca-b7d4-8b4a71b5f80f admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:16:02.355305 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1331/2692] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:16:02.359289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac23495-72ae-4b4f-8beb-3247179df577 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:16:02.369288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac23495-72ae-4b4f-8beb-3247179df577 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:16:02.373918 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc97f6-2970-4037-b684-4b790df6db2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:16:02.374860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc97f6-2970-4037-b684-4b790df6db2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:02.375141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc97f6-2970-4037-b684-4b790df6db2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:02.375413 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc97f6-2970-4037-b684-4b790df6db2f admin admin] PATH_INFO: `/v3/projects/4b59733bf838471f8e92544ed5831d56/users/9f28058bc11e4b28a6b0c25571e8e6e8/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:02.390257 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc97f6-2970-4037-b684-4b790df6db2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b59733bf838471f8e92544ed5831d56, user_id=9f28058bc11e4b28a6b0c25571e8e6e8, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:16:02.392902 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc97f6-2970-4037-b684-4b790df6db2f admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:16:02.480948 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1362/2693] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:16:02 2022] PUT /identity/v3/projects/4b59733bf838471f8e92544ed5831d56/users/9f28058bc11e4b28a6b0c25571e8e6e8/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:16:02.491260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427f00e-41b7-46ea-94d3-a9f2e0788271 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:02.491579 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427f00e-41b7-46ea-94d3-a9f2e0788271 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:02.491877 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427f00e-41b7-46ea-94d3-a9f2e0788271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:02.519355 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-d427f00e-41b7-46ea-94d3-a9f2e0788271 None None] MFA Rules not processed for user `9f28058bc11e4b28a6b0c25571e8e6e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:16:02.520562 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d427f00e-41b7-46ea-94d3-a9f2e0788271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:16:02.583479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1332/2694] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:16:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:16:02.595386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc97f6-2970-4037-b684-4b790df6db2f admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:16:02.597106 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-e1bc97f6-2970-4037-b684-4b790df6db2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:16:02.607237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:16:02.654710 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc97f6-2970-4037-b684-4b790df6db2f admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:16:02.658342 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb38b20-c3f1-451f-8ee4-5082dbacf1ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:16:02.658978 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb38b20-c3f1-451f-8ee4-5082dbacf1ba service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:02.659261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb38b20-c3f1-451f-8ee4-5082dbacf1ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:02.659535 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb38b20-c3f1-451f-8ee4-5082dbacf1ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:02.667936 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb38b20-c3f1-451f-8ee4-5082dbacf1ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:16:02.669332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb38b20-c3f1-451f-8ee4-5082dbacf1ba service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:16:02.677934 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1363/2695] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:16:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:16:12.580694 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=76,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:16:12.695636 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=23 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:16:58.858194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427f00e-41b7-46ea-94d3-a9f2e0788271 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:16:58.919735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427f00e-41b7-46ea-94d3-a9f2e0788271 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:16:58.924965 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adea67f-4c6a-47fc-8218-fdb3ad3f5998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:16:58.926334 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adea67f-4c6a-47fc-8218-fdb3ad3f5998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:58.926334 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adea67f-4c6a-47fc-8218-fdb3ad3f5998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:58.926334 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adea67f-4c6a-47fc-8218-fdb3ad3f5998 admin admin] PATH_INFO: `/v3/users/9f28058bc11e4b28a6b0c25571e8e6e8` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:58.927247 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adea67f-4c6a-47fc-8218-fdb3ad3f5998 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f28058bc11e4b28a6b0c25571e8e6e8)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:16:58.929157 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adea67f-4c6a-47fc-8218-fdb3ad3f5998 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:16:59.108606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1adea67f-4c6a-47fc-8218-fdb3ad3f5998 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f28058bc11e4b28a6b0c25571e8e6e8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:16:59.182278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1adea67f-4c6a-47fc-8218-fdb3ad3f5998 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f28058bc11e4b28a6b0c25571e8e6e8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:16:59.182946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1adea67f-4c6a-47fc-8218-fdb3ad3f5998 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f28058bc11e4b28a6b0c25571e8e6e8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:16:59.195295 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1adea67f-4c6a-47fc-8218-fdb3ad3f5998 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f28058bc11e4b28a6b0c25571e8e6e8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:16:59.201969 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1adea67f-4c6a-47fc-8218-fdb3ad3f5998 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f28058bc11e4b28a6b0c25571e8e6e8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:16:59.210182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1333/2696] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:16:58 2022] DELETE /identity/v3/users/9f28058bc11e4b28a6b0c25571e8e6e8 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:16:59.213274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb104f-44b9-4937-a9b1-40c08436b353 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:59.213768 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb104f-44b9-4937-a9b1-40c08436b353 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:59.213969 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb104f-44b9-4937-a9b1-40c08436b353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:59.233060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-ffbb104f-44b9-4937-a9b1-40c08436b353 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:16:59.237155 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ffbb104f-44b9-4937-a9b1-40c08436b353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:16:59.291476 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1364/2697] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:16:59.301114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adea67f-4c6a-47fc-8218-fdb3ad3f5998 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:16:59.302508 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-1adea67f-4c6a-47fc-8218-fdb3ad3f5998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:16:59.361360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adea67f-4c6a-47fc-8218-fdb3ad3f5998 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:16:59.364879 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3356d32-926d-4cf6-b6a7-90b758656420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:16:59.365480 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3356d32-926d-4cf6-b6a7-90b758656420 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:59.365759 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3356d32-926d-4cf6-b6a7-90b758656420 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:59.366029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3356d32-926d-4cf6-b6a7-90b758656420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:59.374724 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3356d32-926d-4cf6-b6a7-90b758656420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:16:59.376227 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3356d32-926d-4cf6-b6a7-90b758656420 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:16:59.385980 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1334/2698] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:16:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:16:59.816732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb104f-44b9-4937-a9b1-40c08436b353 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:16:59.818139 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ffbb104f-44b9-4937-a9b1-40c08436b353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:16:59.830780 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb104f-44b9-4937-a9b1-40c08436b353 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:16:59.835313 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f170f79a-f423-493c-b683-fe311c366896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:16:59.835976 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170f79a-f423-493c-b683-fe311c366896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:16:59.836270 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170f79a-f423-493c-b683-fe311c366896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:16:59.836540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170f79a-f423-493c-b683-fe311c366896 admin admin] PATH_INFO: `/v3/projects/4b59733bf838471f8e92544ed5831d56` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:16:59.837647 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f170f79a-f423-493c-b683-fe311c366896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b59733bf838471f8e92544ed5831d56)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:16:59.839576 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f170f79a-f423-493c-b683-fe311c366896 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:16:59.951822 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f170f79a-f423-493c-b683-fe311c366896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b59733bf838471f8e92544ed5831d56', 'request_id': 'req-f170f79a-f423-493c-b683-fe311c366896'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:16:59.957761 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f170f79a-f423-493c-b683-fe311c366896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b59733bf838471f8e92544ed5831d56', 'request_id': 'req-f170f79a-f423-493c-b683-fe311c366896'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:17:00.014326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f170f79a-f423-493c-b683-fe311c366896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b59733bf838471f8e92544ed5831d56', 'request_id': 'req-f170f79a-f423-493c-b683-fe311c366896'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:17:00.024627 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f170f79a-f423-493c-b683-fe311c366896 admin admin] The token cache is being invalidate because project 4b59733bf838471f8e92544ed5831d56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:17:00.025019 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f170f79a-f423-493c-b683-fe311c366896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:17:00.025519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f170f79a-f423-493c-b683-fe311c366896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:17:00.027024 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1365/2699] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:16:59 2022] DELETE /identity/v3/projects/4b59733bf838471f8e92544ed5831d56 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:17:00.031973 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0fe8b-9607-4506-a9a3-0be06f35203b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:17:00.032437 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0fe8b-9607-4506-a9a3-0be06f35203b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:17:00.032536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0fe8b-9607-4506-a9a3-0be06f35203b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:17:00.052098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-fac0fe8b-9607-4506-a9a3-0be06f35203b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:17:00.053290 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-fac0fe8b-9607-4506-a9a3-0be06f35203b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:17:00.070047 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:17:00.111503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1335/2700] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:17:00.120504 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c148cd5-f66f-4486-8609-9b1942c2c433 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:17:00.120793 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c148cd5-f66f-4486-8609-9b1942c2c433 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:17:00.121073 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c148cd5-f66f-4486-8609-9b1942c2c433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:17:00.142742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-7c148cd5-f66f-4486-8609-9b1942c2c433 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:17:00.143965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7c148cd5-f66f-4486-8609-9b1942c2c433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:17:00.151778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1366/2701] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:17:00.155952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0fe8b-9607-4506-a9a3-0be06f35203b None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:17:00.165513 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0fe8b-9607-4506-a9a3-0be06f35203b None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:17:00.170179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed1c2f-7aab-4ba5-b52f-89e711667bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:17:00.179260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed1c2f-7aab-4ba5-b52f-89e711667bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:17:00.179260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed1c2f-7aab-4ba5-b52f-89e711667bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:17:00.179260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed1c2f-7aab-4ba5-b52f-89e711667bd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:17:00.179260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed1c2f-7aab-4ba5-b52f-89e711667bd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:17:00.179260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed1c2f-7aab-4ba5-b52f-89e711667bd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:17:00.179260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed1c2f-7aab-4ba5-b52f-89e711667bd6 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:17:00.186574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1336/2702] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 15:17:00.191445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c148cd5-f66f-4486-8609-9b1942c2c433 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:17:00.200586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c148cd5-f66f-4486-8609-9b1942c2c433 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:17:00.205326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befda649-7dd6-4d8e-a21b-4edf710dec50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:17:00.205930 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befda649-7dd6-4d8e-a21b-4edf710dec50 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:17:00.206208 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befda649-7dd6-4d8e-a21b-4edf710dec50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:17:00.206478 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befda649-7dd6-4d8e-a21b-4edf710dec50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:17:00.207155 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befda649-7dd6-4d8e-a21b-4edf710dec50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:17:00.209081 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befda649-7dd6-4d8e-a21b-4edf710dec50 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:17:00.315242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1367/2703] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:17:00 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:17:00.318830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed1c2f-7aab-4ba5-b52f-89e711667bd6 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:17:00.328869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed1c2f-7aab-4ba5-b52f-89e711667bd6 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:17:00.384874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cc2ce-45aa-4708-91be-e5b27d6e7aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:17:00.385469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cc2ce-45aa-4708-91be-e5b27d6e7aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:17:00.385751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cc2ce-45aa-4708-91be-e5b27d6e7aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:17:00.386022 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cc2ce-45aa-4708-91be-e5b27d6e7aa1 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:17:00.386630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3cc2ce-45aa-4708-91be-e5b27d6e7aa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:17:00.388563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3cc2ce-45aa-4708-91be-e5b27d6e7aa1 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:17:00.489149 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1337/2704] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:17:00 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:17:00.493403 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befda649-7dd6-4d8e-a21b-4edf710dec50 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:17:00.503564 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befda649-7dd6-4d8e-a21b-4edf710dec50 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:17:00.508450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd63eb-715a-4707-bfb2-40e7e787608b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:17:00.509039 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd63eb-715a-4707-bfb2-40e7e787608b admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:17:00.509315 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd63eb-715a-4707-bfb2-40e7e787608b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:17:00.509652 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd63eb-715a-4707-bfb2-40e7e787608b admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:17:00.510286 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd63eb-715a-4707-bfb2-40e7e787608b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:17:00.511886 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd63eb-715a-4707-bfb2-40e7e787608b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:17:00.521991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1368/2705] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:17:00.525836 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cc2ce-45aa-4708-91be-e5b27d6e7aa1 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:17:00.536217 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cc2ce-45aa-4708-91be-e5b27d6e7aa1 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:17:00.540853 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1af0f1-21d5-46e1-91f7-1b28ff840dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:17:00.541441 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1af0f1-21d5-46e1-91f7-1b28ff840dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:17:00.541723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1af0f1-21d5-46e1-91f7-1b28ff840dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:17:00.541998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1af0f1-21d5-46e1-91f7-1b28ff840dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:17:00.542633 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1af0f1-21d5-46e1-91f7-1b28ff840dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:17:00.544276 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1af0f1-21d5-46e1-91f7-1b28ff840dd5 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:17:00.555246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1338/2706] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:17:00.559114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd63eb-715a-4707-bfb2-40e7e787608b admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:17:00.568659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd63eb-715a-4707-bfb2-40e7e787608b admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:17:00.573114 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f3133-ee21-49a5-b46f-c1912dc7f525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:17:00.573739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f3133-ee21-49a5-b46f-c1912dc7f525 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:17:00.574020 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f3133-ee21-49a5-b46f-c1912dc7f525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:17:00.574292 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f3133-ee21-49a5-b46f-c1912dc7f525 admin admin] PATH_INFO: `/v3/projects/1128a808f7f64bbcb0934bfe7659b646/users/1540183e55274feb99d7c5ce64896510/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:17:00.588112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f3133-ee21-49a5-b46f-c1912dc7f525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1128a808f7f64bbcb0934bfe7659b646, user_id=1540183e55274feb99d7c5ce64896510, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:17:00.590671 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f3133-ee21-49a5-b46f-c1912dc7f525 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:17:00.706860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1369/2707] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:17:00 2022] PUT /identity/v3/projects/1128a808f7f64bbcb0934bfe7659b646/users/1540183e55274feb99d7c5ce64896510/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:17:00.717428 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5c525-9ff4-4cfc-96a2-b55dce808701 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:17:00.717722 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5c525-9ff4-4cfc-96a2-b55dce808701 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:17:00.717998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5c525-9ff4-4cfc-96a2-b55dce808701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:17:00.746329 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-abe5c525-9ff4-4cfc-96a2-b55dce808701 None None] MFA Rules not processed for user `1540183e55274feb99d7c5ce64896510`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:17:00.747534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-abe5c525-9ff4-4cfc-96a2-b55dce808701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:17:00.810246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1339/2708] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:17:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:17:00.821341 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f3133-ee21-49a5-b46f-c1912dc7f525 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:17:00.823089 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b83f3133-ee21-49a5-b46f-c1912dc7f525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:17:00.845076 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:17:00.878320 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f3133-ee21-49a5-b46f-c1912dc7f525 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:17:00.882563 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785988d-068d-403c-89a6-a1b6538f9b6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:17:00.883447 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785988d-068d-403c-89a6-a1b6538f9b6f service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:17:00.883725 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785988d-068d-403c-89a6-a1b6538f9b6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:17:00.883992 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785988d-068d-403c-89a6-a1b6538f9b6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:17:00.892706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785988d-068d-403c-89a6-a1b6538f9b6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:17:00.894138 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785988d-068d-403c-89a6-a1b6538f9b6f service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:17:00.902886 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1370/2709] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:17:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:17:00.928128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5c525-9ff4-4cfc-96a2-b55dce808701 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:17:00.946474 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:17:00.989261 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5c525-9ff4-4cfc-96a2-b55dce808701 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:17:00.993902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47989868-3eb3-4dd0-bd5c-3eb1aaa1e59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:17:00.994532 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47989868-3eb3-4dd0-bd5c-3eb1aaa1e59a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:17:00.994836 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47989868-3eb3-4dd0-bd5c-3eb1aaa1e59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:17:00.995115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47989868-3eb3-4dd0-bd5c-3eb1aaa1e59a admin admin] PATH_INFO: `/v3/users/1540183e55274feb99d7c5ce64896510` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:17:00.996283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47989868-3eb3-4dd0-bd5c-3eb1aaa1e59a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1540183e55274feb99d7c5ce64896510)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:17:00.998224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47989868-3eb3-4dd0-bd5c-3eb1aaa1e59a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:17:01.133483 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-47989868-3eb3-4dd0-bd5c-3eb1aaa1e59a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1540183e55274feb99d7c5ce64896510'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:17:01.205516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-47989868-3eb3-4dd0-bd5c-3eb1aaa1e59a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1540183e55274feb99d7c5ce64896510'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:17:01.206150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-47989868-3eb3-4dd0-bd5c-3eb1aaa1e59a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1540183e55274feb99d7c5ce64896510'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:17:01.218845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-47989868-3eb3-4dd0-bd5c-3eb1aaa1e59a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1540183e55274feb99d7c5ce64896510'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:17:01.225744 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-47989868-3eb3-4dd0-bd5c-3eb1aaa1e59a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1540183e55274feb99d7c5ce64896510'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:17:01.231647 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1340/2710] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:17:00 2022] DELETE /identity/v3/users/1540183e55274feb99d7c5ce64896510 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:17:01.236692 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9040395-60fa-41f5-a621-9926c420e4c7 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:17:01.236814 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9040395-60fa-41f5-a621-9926c420e4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:17:01.237004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9040395-60fa-41f5-a621-9926c420e4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:17:01.256078 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-c9040395-60fa-41f5-a621-9926c420e4c7 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:17:01.257274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c9040395-60fa-41f5-a621-9926c420e4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:17:01.317159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1371/2711] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:17:01.327183 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47989868-3eb3-4dd0-bd5c-3eb1aaa1e59a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:17:01.329235 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-47989868-3eb3-4dd0-bd5c-3eb1aaa1e59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:17:01.385953 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47989868-3eb3-4dd0-bd5c-3eb1aaa1e59a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:17:01.389539 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59057836-7cce-4325-be93-1d9da2031f2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:17:01.390138 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59057836-7cce-4325-be93-1d9da2031f2d service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:17:01.390415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59057836-7cce-4325-be93-1d9da2031f2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:17:01.391561 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59057836-7cce-4325-be93-1d9da2031f2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:17:01.399897 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59057836-7cce-4325-be93-1d9da2031f2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:17:01.401281 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59057836-7cce-4325-be93-1d9da2031f2d service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:17:01.411358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1341/2712] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 15:17:01.817273 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9040395-60fa-41f5-a621-9926c420e4c7 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:17:01.818696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c9040395-60fa-41f5-a621-9926c420e4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:17:01.836036 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9040395-60fa-41f5-a621-9926c420e4c7 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:17:01.840772 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:17:01.841591 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:17:01.841920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:17:01.842128 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3 admin admin] PATH_INFO: `/v3/projects/1128a808f7f64bbcb0934bfe7659b646` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:17:01.843280 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1128a808f7f64bbcb0934bfe7659b646)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:17:01.845175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:17:01.983221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1128a808f7f64bbcb0934bfe7659b646', 'request_id': 'req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:17:01.989369 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1128a808f7f64bbcb0934bfe7659b646', 'request_id': 'req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:17:02.056180 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1128a808f7f64bbcb0934bfe7659b646', 'request_id': 'req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:17:02.067248 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3 admin admin] The token cache is being invalidate because project 1128a808f7f64bbcb0934bfe7659b646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:17:02.067480 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:17:02.068093 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:17:02.069560 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1372/2713] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:17:01 2022] DELETE /identity/v3/projects/1128a808f7f64bbcb0934bfe7659b646 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:17:11.407796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:17:12.063938 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:17:56.347623 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f870137-8ed5-4b09-9880-c1cad6c79ee6 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:17:56.348147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f870137-8ed5-4b09-9880-c1cad6c79ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:17:56.348147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f870137-8ed5-4b09-9880-c1cad6c79ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:17:56.376389 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-9f870137-8ed5-4b09-9880-c1cad6c79ee6 None None] MFA Rules not processed for user `3ad73a36c45a4763b00027457fb4117f`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:17:56.378553 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9f870137-8ed5-4b09-9880-c1cad6c79ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:17:56.450984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1342/2714] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:17:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 15:18:01.771061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:18:01.773287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:18:01.835473 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fde0a5-70b3-4ce4-bb5c-e3ad57ade2c3 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:18:01.839117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a2da7-100f-4f71-8260-49c61ba15e57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:18:01.839721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a2da7-100f-4f71-8260-49c61ba15e57 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:18:01.840001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a2da7-100f-4f71-8260-49c61ba15e57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:18:01.840272 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a2da7-100f-4f71-8260-49c61ba15e57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:18:01.842026 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d51a2da7-100f-4f71-8260-49c61ba15e57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:18:01.922848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a2da7-100f-4f71-8260-49c61ba15e57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:18:01.924275 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a2da7-100f-4f71-8260-49c61ba15e57 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:18:01.933936 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1373/2715] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:18:01 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:18:06.444323 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=25 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:18:11.929763 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=57 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:18:12.236121 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f870137-8ed5-4b09-9880-c1cad6c79ee6 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:18:12.237559 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9f870137-8ed5-4b09-9880-c1cad6c79ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:18:12.314651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f870137-8ed5-4b09-9880-c1cad6c79ee6 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:18:12.320255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665aeec-09e3-4ded-9f50-3cdcf553e29a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:18:12.320869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665aeec-09e3-4ded-9f50-3cdcf553e29a service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:18:12.321165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665aeec-09e3-4ded-9f50-3cdcf553e29a service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:18:12.321418 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665aeec-09e3-4ded-9f50-3cdcf553e29a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:18:12.323237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-3665aeec-09e3-4ded-9f50-3cdcf553e29a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:18:12.335657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665aeec-09e3-4ded-9f50-3cdcf553e29a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:18:12.337283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665aeec-09e3-4ded-9f50-3cdcf553e29a service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:18:12.349011 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1343/2716] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:18:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:18:22.343707 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=34 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:19:04.527316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a2da7-100f-4f71-8260-49c61ba15e57 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:19:04.528689 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d51a2da7-100f-4f71-8260-49c61ba15e57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:19:04.608528 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a2da7-100f-4f71-8260-49c61ba15e57 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:19:04.613523 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687e851-b259-4c82-b969-52340734680c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:19:04.614132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687e851-b259-4c82-b969-52340734680c service placement] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:19:04.614416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687e851-b259-4c82-b969-52340734680c service placement] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:19:04.614693 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687e851-b259-4c82-b969-52340734680c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:19:04.617079 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c687e851-b259-4c82-b969-52340734680c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:19:04.628203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687e851-b259-4c82-b969-52340734680c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:19:04.629763 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687e851-b259-4c82-b969-52340734680c service placement] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:19:04.639770 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1374/2717] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:19:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:19:14.635636 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=37 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:19:25.054921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665aeec-09e3-4ded-9f50-3cdcf553e29a service nova] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:19:25.059610 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-3665aeec-09e3-4ded-9f50-3cdcf553e29a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:19:25.082632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665aeec-09e3-4ded-9f50-3cdcf553e29a service nova] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:19:25.088793 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8385854-6098-430c-b9ab-33deea512fc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:19:25.089397 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8385854-6098-430c-b9ab-33deea512fc4 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:19:25.089685 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8385854-6098-430c-b9ab-33deea512fc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:19:25.090051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8385854-6098-430c-b9ab-33deea512fc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:19:25.091796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e8385854-6098-430c-b9ab-33deea512fc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:19:25.102242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8385854-6098-430c-b9ab-33deea512fc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:19:25.103840 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8385854-6098-430c-b9ab-33deea512fc4 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:19:25.112558 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1344/2718] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:19:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:19:35.108478 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=11 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:19:53.816611 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687e851-b259-4c82-b969-52340734680c service placement] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:19:53.829997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687e851-b259-4c82-b969-52340734680c service placement] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:19:53.835623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568010c3-9aa0-4422-bc45-bfdb4839cb3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:19:53.836231 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568010c3-9aa0-4422-bc45-bfdb4839cb3a service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:19:53.836511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568010c3-9aa0-4422-bc45-bfdb4839cb3a service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:19:53.836784 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568010c3-9aa0-4422-bc45-bfdb4839cb3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:19:53.838530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-568010c3-9aa0-4422-bc45-bfdb4839cb3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:19:53.850940 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568010c3-9aa0-4422-bc45-bfdb4839cb3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:19:53.852515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568010c3-9aa0-4422-bc45-bfdb4839cb3a service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:19:53.863429 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1375/2719] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:19:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:20:00.203124 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8385854-6098-430c-b9ab-33deea512fc4 service nova] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:00.204514 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e8385854-6098-430c-b9ab-33deea512fc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:00.259070 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8385854-6098-430c-b9ab-33deea512fc4 service nova] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:00.263424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256b3f5-cffd-4732-8586-1f35910b70da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:00.264025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256b3f5-cffd-4732-8586-1f35910b70da service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:00.264309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256b3f5-cffd-4732-8586-1f35910b70da service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:00.264584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256b3f5-cffd-4732-8586-1f35910b70da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:00.266300 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e256b3f5-cffd-4732-8586-1f35910b70da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:00.326600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256b3f5-cffd-4732-8586-1f35910b70da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:00.328057 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256b3f5-cffd-4732-8586-1f35910b70da service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:00.337385 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1345/2720] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:20:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:20:03.736616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568010c3-9aa0-4422-bc45-bfdb4839cb3a service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:03.749676 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568010c3-9aa0-4422-bc45-bfdb4839cb3a service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:03.755083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf729df-4d45-41d7-8a4a-fb742396d0a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:03.755795 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf729df-4d45-41d7-8a4a-fb742396d0a1 service placement] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:03.755964 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf729df-4d45-41d7-8a4a-fb742396d0a1 service placement] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:03.756416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf729df-4d45-41d7-8a4a-fb742396d0a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:03.758190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-baf729df-4d45-41d7-8a4a-fb742396d0a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:03.775283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf729df-4d45-41d7-8a4a-fb742396d0a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:03.776911 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf729df-4d45-41d7-8a4a-fb742396d0a1 service placement] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:03.790757 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1376/2721] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:20:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:20:03.872248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256b3f5-cffd-4732-8586-1f35910b70da service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:03.873836 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e256b3f5-cffd-4732-8586-1f35910b70da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:03.887315 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256b3f5-cffd-4732-8586-1f35910b70da service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:03.892190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81534cfb-3699-4c8a-b9b4-e79d0b46670b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:03.892834 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81534cfb-3699-4c8a-b9b4-e79d0b46670b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:03.893269 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81534cfb-3699-4c8a-b9b4-e79d0b46670b admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:03.893844 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81534cfb-3699-4c8a-b9b4-e79d0b46670b admin admin] PATH_INFO: `/v3/users/3ad73a36c45a4763b00027457fb4117f` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:03.895170 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81534cfb-3699-4c8a-b9b4-e79d0b46670b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ad73a36c45a4763b00027457fb4117f)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:03.897165 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81534cfb-3699-4c8a-b9b4-e79d0b46670b admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:04.066958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-81534cfb-3699-4c8a-b9b4-e79d0b46670b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ad73a36c45a4763b00027457fb4117f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:04.127964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-81534cfb-3699-4c8a-b9b4-e79d0b46670b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ad73a36c45a4763b00027457fb4117f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:04.128522 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-81534cfb-3699-4c8a-b9b4-e79d0b46670b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ad73a36c45a4763b00027457fb4117f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:04.141299 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-81534cfb-3699-4c8a-b9b4-e79d0b46670b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ad73a36c45a4763b00027457fb4117f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:04.148121 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-81534cfb-3699-4c8a-b9b4-e79d0b46670b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ad73a36c45a4763b00027457fb4117f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:04.154457 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1346/2722] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:20:03 2022] DELETE /identity/v3/users/3ad73a36c45a4763b00027457fb4117f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:04.432141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf729df-4d45-41d7-8a4a-fb742396d0a1 service placement] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:04.442753 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf729df-4d45-41d7-8a4a-fb742396d0a1 service placement] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:04.503608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb4da4-876f-4179-b91a-240a77934a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:04.504390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb4da4-876f-4179-b91a-240a77934a20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:04.504679 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb4da4-876f-4179-b91a-240a77934a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:04.504954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb4da4-876f-4179-b91a-240a77934a20 admin admin] PATH_INFO: `/v3/projects/7c858eb651e44140960444974d8bc4c4` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:04.506065 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb4da4-876f-4179-b91a-240a77934a20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c858eb651e44140960444974d8bc4c4)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:04.508005 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb4da4-876f-4179-b91a-240a77934a20 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:04.590917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-06bb4da4-876f-4179-b91a-240a77934a20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c858eb651e44140960444974d8bc4c4', 'request_id': 'req-06bb4da4-876f-4179-b91a-240a77934a20'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:04.597202 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-06bb4da4-876f-4179-b91a-240a77934a20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c858eb651e44140960444974d8bc4c4', 'request_id': 'req-06bb4da4-876f-4179-b91a-240a77934a20'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:04.653421 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-06bb4da4-876f-4179-b91a-240a77934a20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c858eb651e44140960444974d8bc4c4', 'request_id': 'req-06bb4da4-876f-4179-b91a-240a77934a20'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:04.664084 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-06bb4da4-876f-4179-b91a-240a77934a20 admin admin] The token cache is being invalidate because project 7c858eb651e44140960444974d8bc4c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:20:04.664371 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-06bb4da4-876f-4179-b91a-240a77934a20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:04.664992 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-06bb4da4-876f-4179-b91a-240a77934a20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:04.666518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1377/2723] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:20:04 2022] DELETE /identity/v3/projects/7c858eb651e44140960444974d8bc4c4 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:04.671521 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e446f67-d1ba-4d26-ba7e-22660424d63b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:04.671807 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e446f67-d1ba-4d26-ba7e-22660424d63b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:04.672084 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e446f67-d1ba-4d26-ba7e-22660424d63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:04.691491 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-3e446f67-d1ba-4d26-ba7e-22660424d63b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:04.692671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-3e446f67-d1ba-4d26-ba7e-22660424d63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:04.711037 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:04.752754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1347/2724] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:04.760626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447546a-ebd5-45f6-87c5-a741906db7c9 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:04.761891 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447546a-ebd5-45f6-87c5-a741906db7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:04.762020 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447546a-ebd5-45f6-87c5-a741906db7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:04.782009 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-1447546a-ebd5-45f6-87c5-a741906db7c9 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:04.783363 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-1447546a-ebd5-45f6-87c5-a741906db7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:04.791293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1378/2725] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:04.795523 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e446f67-d1ba-4d26-ba7e-22660424d63b None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:04.805144 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e446f67-d1ba-4d26-ba7e-22660424d63b None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:04.809764 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317b570-527a-411e-8368-9f51f5962409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:04.810349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317b570-527a-411e-8368-9f51f5962409 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:04.810626 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317b570-527a-411e-8368-9f51f5962409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:04.810939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317b570-527a-411e-8368-9f51f5962409 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:04.811582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317b570-527a-411e-8368-9f51f5962409 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:20:04.811884 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317b570-527a-411e-8368-9f51f5962409 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:04.813586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317b570-527a-411e-8368-9f51f5962409 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:04.825336 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1348/2726] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 15:20:04.829554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447546a-ebd5-45f6-87c5-a741906db7c9 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:04.838699 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447546a-ebd5-45f6-87c5-a741906db7c9 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:04.844749 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0858d73-4ed2-4377-b80e-9245f7c27a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:04.846595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0858d73-4ed2-4377-b80e-9245f7c27a11 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:04.846595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0858d73-4ed2-4377-b80e-9245f7c27a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:04.846595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0858d73-4ed2-4377-b80e-9245f7c27a11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:04.846595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0858d73-4ed2-4377-b80e-9245f7c27a11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:04.848540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0858d73-4ed2-4377-b80e-9245f7c27a11 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:04.957598 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1379/2727] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:20:04 2022] POST /identity/v3/projects => generated 357 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:20:04.961813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317b570-527a-411e-8368-9f51f5962409 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:04.971746 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317b570-527a-411e-8368-9f51f5962409 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:05.028661 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662a2f8-770a-41c6-ba3b-0e96164395d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:05.029266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662a2f8-770a-41c6-ba3b-0e96164395d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:05.029550 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662a2f8-770a-41c6-ba3b-0e96164395d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:05.029825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662a2f8-770a-41c6-ba3b-0e96164395d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:05.030448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662a2f8-770a-41c6-ba3b-0e96164395d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:05.032420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662a2f8-770a-41c6-ba3b-0e96164395d4 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:05.099622 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1349/2728] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:20:04 2022] POST /identity/v3/users => generated 279 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:20:05.106293 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0858d73-4ed2-4377-b80e-9245f7c27a11 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:05.114221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0858d73-4ed2-4377-b80e-9245f7c27a11 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:05.119030 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0b643-74c3-4186-b2f9-a6af19118c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:05.119624 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0b643-74c3-4186-b2f9-a6af19118c21 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:05.119909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0b643-74c3-4186-b2f9-a6af19118c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:05.120681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0b643-74c3-4186-b2f9-a6af19118c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:05.121353 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0b643-74c3-4186-b2f9-a6af19118c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:05.122996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0b643-74c3-4186-b2f9-a6af19118c21 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:05.132969 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1380/2729] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 15:20:05.137952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662a2f8-770a-41c6-ba3b-0e96164395d4 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:05.147914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662a2f8-770a-41c6-ba3b-0e96164395d4 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:05.152471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59daf2a-c51a-41ce-8a56-4c6aea044c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:05.153066 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59daf2a-c51a-41ce-8a56-4c6aea044c69 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:05.153351 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59daf2a-c51a-41ce-8a56-4c6aea044c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:05.153624 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59daf2a-c51a-41ce-8a56-4c6aea044c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:05.154261 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59daf2a-c51a-41ce-8a56-4c6aea044c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:05.155895 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59daf2a-c51a-41ce-8a56-4c6aea044c69 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:05.166534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1350/2730] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:05.170412 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0b643-74c3-4186-b2f9-a6af19118c21 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:05.180170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0b643-74c3-4186-b2f9-a6af19118c21 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:05.185276 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8fe17-6d45-4e37-b561-6314232b7385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:05.185909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8fe17-6d45-4e37-b561-6314232b7385 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:05.186190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8fe17-6d45-4e37-b561-6314232b7385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:05.186466 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8fe17-6d45-4e37-b561-6314232b7385 admin admin] PATH_INFO: `/v3/projects/9d4d12c794ed49f4a6041d0d46a3d643/users/b64b5ffe9d9548b097fb0d93622fe59f/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:05.201204 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8fe17-6d45-4e37-b561-6314232b7385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d4d12c794ed49f4a6041d0d46a3d643, user_id=b64b5ffe9d9548b097fb0d93622fe59f, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:05.203853 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8fe17-6d45-4e37-b561-6314232b7385 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:05.271719 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:05.276331 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1381/2731] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:20:05 2022] PUT /identity/v3/projects/9d4d12c794ed49f4a6041d0d46a3d643/users/b64b5ffe9d9548b097fb0d93622fe59f/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:20:05.286471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d0aa6-6f56-433a-b06e-56f8bebb9823 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:05.286763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d0aa6-6f56-433a-b06e-56f8bebb9823 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:05.287076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d0aa6-6f56-433a-b06e-56f8bebb9823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:05.315510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-fa3d0aa6-6f56-433a-b06e-56f8bebb9823 None None] MFA Rules not processed for user `b64b5ffe9d9548b097fb0d93622fe59f`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:05.316723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-fa3d0aa6-6f56-433a-b06e-56f8bebb9823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:05.382207 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1351/2732] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:05 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:05.387493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8fe17-6d45-4e37-b561-6314232b7385 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:05.447368 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8fe17-6d45-4e37-b561-6314232b7385 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:05.452706 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f9e33-7d1a-4456-b360-dacbdac90fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:05.453308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f9e33-7d1a-4456-b360-dacbdac90fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:05.453947 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f9e33-7d1a-4456-b360-dacbdac90fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:05.453947 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f9e33-7d1a-4456-b360-dacbdac90fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:05.454440 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f9e33-7d1a-4456-b360-dacbdac90fb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:05.456518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f9e33-7d1a-4456-b360-dacbdac90fb1 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:05.551191 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1382/2733] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:20:05 2022] POST /identity/v3/projects => generated 357 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:20:05.555256 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d0aa6-6f56-433a-b06e-56f8bebb9823 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:05.565994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d0aa6-6f56-433a-b06e-56f8bebb9823 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:05.578609 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:05.623325 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43216d-2f25-4801-b148-fd07542ce212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:05.623927 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43216d-2f25-4801-b148-fd07542ce212 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:05.624213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43216d-2f25-4801-b148-fd07542ce212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:05.624487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43216d-2f25-4801-b148-fd07542ce212 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:05.625102 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43216d-2f25-4801-b148-fd07542ce212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:05.627041 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43216d-2f25-4801-b148-fd07542ce212 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:05.727166 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1352/2734] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:20:05 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:20:05.731234 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f9e33-7d1a-4456-b360-dacbdac90fb1 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:05.741659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f9e33-7d1a-4456-b360-dacbdac90fb1 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:05.747250 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc23bb9-e9ff-4408-a9c2-bd21c8f3ed56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:05.747885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc23bb9-e9ff-4408-a9c2-bd21c8f3ed56 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:05.748170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc23bb9-e9ff-4408-a9c2-bd21c8f3ed56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:05.748471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc23bb9-e9ff-4408-a9c2-bd21c8f3ed56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:05.749179 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc23bb9-e9ff-4408-a9c2-bd21c8f3ed56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:05.750945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc23bb9-e9ff-4408-a9c2-bd21c8f3ed56 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:05.763662 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1383/2735] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 15:20:05.767633 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43216d-2f25-4801-b148-fd07542ce212 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:05.777871 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43216d-2f25-4801-b148-fd07542ce212 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:05.782575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6a0e8-5cdc-4dd3-9394-b4fa0a1d99da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:05.783258 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6a0e8-5cdc-4dd3-9394-b4fa0a1d99da admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:05.783540 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6a0e8-5cdc-4dd3-9394-b4fa0a1d99da admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:05.783819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6a0e8-5cdc-4dd3-9394-b4fa0a1d99da admin admin] PATH_INFO: `/v3/projects/309fff014a4643598483358140dcd2d9/users/fa53c0575b684df083d86e3bdbf17073/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:05.798408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6a0e8-5cdc-4dd3-9394-b4fa0a1d99da admin admin] RBAC: Authorizing `identity:create_grant(project_id=309fff014a4643598483358140dcd2d9, user_id=fa53c0575b684df083d86e3bdbf17073, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:05.801073 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6a0e8-5cdc-4dd3-9394-b4fa0a1d99da admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:05.864388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1353/2736] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:20:05 2022] PUT /identity/v3/projects/309fff014a4643598483358140dcd2d9/users/fa53c0575b684df083d86e3bdbf17073/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:05.868204 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc23bb9-e9ff-4408-a9c2-bd21c8f3ed56 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:05.878854 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc23bb9-e9ff-4408-a9c2-bd21c8f3ed56 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:05.934326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b3795-71a1-4f37-9765-36b8a33464e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:05.935034 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b3795-71a1-4f37-9765-36b8a33464e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:05.935316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b3795-71a1-4f37-9765-36b8a33464e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:05.935590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b3795-71a1-4f37-9765-36b8a33464e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:05.936222 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b3795-71a1-4f37-9765-36b8a33464e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:05.937819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b3795-71a1-4f37-9765-36b8a33464e7 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:05.947227 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1384/2737] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:05.952006 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6a0e8-5cdc-4dd3-9394-b4fa0a1d99da admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:05.962157 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6a0e8-5cdc-4dd3-9394-b4fa0a1d99da admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:05.966695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a310af1-12a2-47e7-b838-a739be1469ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:05.967363 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a310af1-12a2-47e7-b838-a739be1469ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:05.967641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a310af1-12a2-47e7-b838-a739be1469ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:05.967913 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a310af1-12a2-47e7-b838-a739be1469ee admin admin] PATH_INFO: `/v3/projects/309fff014a4643598483358140dcd2d9/users/fa53c0575b684df083d86e3bdbf17073/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:05.969798 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a310af1-12a2-47e7-b838-a739be1469ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=309fff014a4643598483358140dcd2d9, user_id=fa53c0575b684df083d86e3bdbf17073, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:05.972470 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a310af1-12a2-47e7-b838-a739be1469ee admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:05.983003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1354/2738] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:20:05 2022] PUT /identity/v3/projects/309fff014a4643598483358140dcd2d9/users/fa53c0575b684df083d86e3bdbf17073/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:05.993828 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5860964-a7c1-494e-90fa-5f5f313f6f30 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:05.994115 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5860964-a7c1-494e-90fa-5f5f313f6f30 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:05.994394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5860964-a7c1-494e-90fa-5f5f313f6f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:06.022554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-b5860964-a7c1-494e-90fa-5f5f313f6f30 None None] MFA Rules not processed for user `fa53c0575b684df083d86e3bdbf17073`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:06.023803 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b5860964-a7c1-494e-90fa-5f5f313f6f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:06.081724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:06.090515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1385/2739] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:05 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:06.101682 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a310af1-12a2-47e7-b838-a739be1469ee admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:06.103486 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-1a310af1-12a2-47e7-b838-a739be1469ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:06.160940 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a310af1-12a2-47e7-b838-a739be1469ee admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:06.164466 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e42c31-b30d-4c58-8ab8-d8a758672993 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:06.165086 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e42c31-b30d-4c58-8ab8-d8a758672993 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:06.165366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e42c31-b30d-4c58-8ab8-d8a758672993 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:06.165637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e42c31-b30d-4c58-8ab8-d8a758672993 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:06.174916 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e42c31-b30d-4c58-8ab8-d8a758672993 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:06.176340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e42c31-b30d-4c58-8ab8-d8a758672993 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:06.186077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1355/2740] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:20:06 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:20:06.369850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5860964-a7c1-494e-90fa-5f5f313f6f30 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:06.380004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5860964-a7c1-494e-90fa-5f5f313f6f30 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:06.383986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4faff-b930-44ce-92d4-6304f71f3403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:06.387590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4faff-b930-44ce-92d4-6304f71f3403 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:06.387590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4faff-b930-44ce-92d4-6304f71f3403 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:06.387590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4faff-b930-44ce-92d4-6304f71f3403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:06.442317 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e4faff-b930-44ce-92d4-6304f71f3403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:06.443696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e4faff-b930-44ce-92d4-6304f71f3403 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:06.452516 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1386/2741] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:20:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:20:16.181352 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:16.448551 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=30 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:24.604220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e42c31-b30d-4c58-8ab8-d8a758672993 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:24.670325 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e42c31-b30d-4c58-8ab8-d8a758672993 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:24.674863 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845a37d-45fd-41e4-98e2-10bc6431815c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:24.675490 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845a37d-45fd-41e4-98e2-10bc6431815c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:24.675772 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845a37d-45fd-41e4-98e2-10bc6431815c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:24.676046 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845a37d-45fd-41e4-98e2-10bc6431815c admin admin] PATH_INFO: `/v3/users/b64b5ffe9d9548b097fb0d93622fe59f` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:24.677167 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845a37d-45fd-41e4-98e2-10bc6431815c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b64b5ffe9d9548b097fb0d93622fe59f)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:24.679135 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845a37d-45fd-41e4-98e2-10bc6431815c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:24.850284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1845a37d-45fd-41e4-98e2-10bc6431815c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b64b5ffe9d9548b097fb0d93622fe59f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:24.909883 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1845a37d-45fd-41e4-98e2-10bc6431815c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b64b5ffe9d9548b097fb0d93622fe59f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:24.910495 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1845a37d-45fd-41e4-98e2-10bc6431815c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b64b5ffe9d9548b097fb0d93622fe59f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:24.923913 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1845a37d-45fd-41e4-98e2-10bc6431815c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b64b5ffe9d9548b097fb0d93622fe59f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:24.931419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-1845a37d-45fd-41e4-98e2-10bc6431815c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b64b5ffe9d9548b097fb0d93622fe59f'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:24.938173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1356/2742] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:20:24 2022] DELETE /identity/v3/users/b64b5ffe9d9548b097fb0d93622fe59f => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:24.942638 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32e61e-014c-4130-b220-9c9ffa16a552 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:24.943223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32e61e-014c-4130-b220-9c9ffa16a552 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:24.943330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32e61e-014c-4130-b220-9c9ffa16a552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:24.962283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-9c32e61e-014c-4130-b220-9c9ffa16a552 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:24.963507 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-9c32e61e-014c-4130-b220-9c9ffa16a552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:25.021602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1387/2743] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:25.031381 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845a37d-45fd-41e4-98e2-10bc6431815c admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:25.032765 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-1845a37d-45fd-41e4-98e2-10bc6431815c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:25.090332 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845a37d-45fd-41e4-98e2-10bc6431815c admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:25.094541 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1dbd2-169b-4898-99bc-ea0c753c355f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:25.095181 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1dbd2-169b-4898-99bc-ea0c753c355f service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:25.095464 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1dbd2-169b-4898-99bc-ea0c753c355f service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:25.095738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1dbd2-169b-4898-99bc-ea0c753c355f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:25.104789 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1dbd2-169b-4898-99bc-ea0c753c355f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:25.106163 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1dbd2-169b-4898-99bc-ea0c753c355f service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:25.116080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1357/2744] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:20:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:20:25.472381 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32e61e-014c-4130-b220-9c9ffa16a552 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:25.478109 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-9c32e61e-014c-4130-b220-9c9ffa16a552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:25.487846 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32e61e-014c-4130-b220-9c9ffa16a552 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:25.492431 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b97716-9d7b-4408-96f8-f1d095a36a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:25.493687 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b97716-9d7b-4408-96f8-f1d095a36a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:25.493985 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b97716-9d7b-4408-96f8-f1d095a36a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:25.494445 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b97716-9d7b-4408-96f8-f1d095a36a80 admin admin] PATH_INFO: `/v3/projects/9d4d12c794ed49f4a6041d0d46a3d643` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:25.495641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b97716-9d7b-4408-96f8-f1d095a36a80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d4d12c794ed49f4a6041d0d46a3d643)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:25.497497 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b97716-9d7b-4408-96f8-f1d095a36a80 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:25.578220 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-51b97716-9d7b-4408-96f8-f1d095a36a80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d4d12c794ed49f4a6041d0d46a3d643', 'request_id': 'req-51b97716-9d7b-4408-96f8-f1d095a36a80'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:25.584519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-51b97716-9d7b-4408-96f8-f1d095a36a80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d4d12c794ed49f4a6041d0d46a3d643', 'request_id': 'req-51b97716-9d7b-4408-96f8-f1d095a36a80'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:25.635815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-51b97716-9d7b-4408-96f8-f1d095a36a80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d4d12c794ed49f4a6041d0d46a3d643', 'request_id': 'req-51b97716-9d7b-4408-96f8-f1d095a36a80'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:25.646998 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-51b97716-9d7b-4408-96f8-f1d095a36a80 admin admin] The token cache is being invalidate because project 9d4d12c794ed49f4a6041d0d46a3d643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:20:25.647286 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-51b97716-9d7b-4408-96f8-f1d095a36a80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:25.647921 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-51b97716-9d7b-4408-96f8-f1d095a36a80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:25.649389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1388/2745] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:20:25 2022] DELETE /identity/v3/projects/9d4d12c794ed49f4a6041d0d46a3d643 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:25.653340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1dbd2-169b-4898-99bc-ea0c753c355f service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:25.663837 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1dbd2-169b-4898-99bc-ea0c753c355f service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:25.681482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:25.719694 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70d3d7-9a6a-40b8-ad23-8817b238db6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:25.720359 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70d3d7-9a6a-40b8-ad23-8817b238db6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:25.720639 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70d3d7-9a6a-40b8-ad23-8817b238db6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:25.720912 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70d3d7-9a6a-40b8-ad23-8817b238db6c admin admin] PATH_INFO: `/v3/users/fa53c0575b684df083d86e3bdbf17073` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:25.722009 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70d3d7-9a6a-40b8-ad23-8817b238db6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa53c0575b684df083d86e3bdbf17073)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:25.723992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70d3d7-9a6a-40b8-ad23-8817b238db6c admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:25.854643 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4f70d3d7-9a6a-40b8-ad23-8817b238db6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa53c0575b684df083d86e3bdbf17073'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:25.916063 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4f70d3d7-9a6a-40b8-ad23-8817b238db6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa53c0575b684df083d86e3bdbf17073'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:25.916683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4f70d3d7-9a6a-40b8-ad23-8817b238db6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa53c0575b684df083d86e3bdbf17073'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:25.929986 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4f70d3d7-9a6a-40b8-ad23-8817b238db6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa53c0575b684df083d86e3bdbf17073'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:25.937412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4f70d3d7-9a6a-40b8-ad23-8817b238db6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa53c0575b684df083d86e3bdbf17073'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:25.943246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1358/2746] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:20:25 2022] DELETE /identity/v3/users/fa53c0575b684df083d86e3bdbf17073 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:26.346028 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b97716-9d7b-4408-96f8-f1d095a36a80 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:26.347603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-51b97716-9d7b-4408-96f8-f1d095a36a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:26.362723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b97716-9d7b-4408-96f8-f1d095a36a80 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:26.419246 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9099d21-bc3b-45d1-a775-3a858bb5a22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:26.419805 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9099d21-bc3b-45d1-a775-3a858bb5a22f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:26.420085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9099d21-bc3b-45d1-a775-3a858bb5a22f admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:26.421185 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9099d21-bc3b-45d1-a775-3a858bb5a22f admin admin] PATH_INFO: `/v3/projects/309fff014a4643598483358140dcd2d9` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:26.422086 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9099d21-bc3b-45d1-a775-3a858bb5a22f admin admin] RBAC: Authorizing `identity:delete_project(project_id=309fff014a4643598483358140dcd2d9)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:26.424070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9099d21-bc3b-45d1-a775-3a858bb5a22f admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:26.505879 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:26.521744 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a9099d21-bc3b-45d1-a775-3a858bb5a22f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '309fff014a4643598483358140dcd2d9', 'request_id': 'req-a9099d21-bc3b-45d1-a775-3a858bb5a22f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:26.527822 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a9099d21-bc3b-45d1-a775-3a858bb5a22f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '309fff014a4643598483358140dcd2d9', 'request_id': 'req-a9099d21-bc3b-45d1-a775-3a858bb5a22f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:26.584116 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a9099d21-bc3b-45d1-a775-3a858bb5a22f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '309fff014a4643598483358140dcd2d9', 'request_id': 'req-a9099d21-bc3b-45d1-a775-3a858bb5a22f'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:26.595434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a9099d21-bc3b-45d1-a775-3a858bb5a22f admin admin] The token cache is being invalidate because project 309fff014a4643598483358140dcd2d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:20:26.595729 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a9099d21-bc3b-45d1-a775-3a858bb5a22f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:26.596347 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-a9099d21-bc3b-45d1-a775-3a858bb5a22f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:26.597826 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1389/2747] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:20:26 2022] DELETE /identity/v3/projects/309fff014a4643598483358140dcd2d9 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:26.602688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724d34e-97aa-4634-9612-14f809004048 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:26.603002 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724d34e-97aa-4634-9612-14f809004048 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:26.603280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724d34e-97aa-4634-9612-14f809004048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:26.623519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-3724d34e-97aa-4634-9612-14f809004048 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:26.624708 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-3724d34e-97aa-4634-9612-14f809004048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:26.683659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1359/2748] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:26.691175 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff25ef5-9935-47b1-bc07-63c8bfe33637 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:26.691463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff25ef5-9935-47b1-bc07-63c8bfe33637 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:26.691734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff25ef5-9935-47b1-bc07-63c8bfe33637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:26.711001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-1ff25ef5-9935-47b1-bc07-63c8bfe33637 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:26.712202 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-1ff25ef5-9935-47b1-bc07-63c8bfe33637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:26.720456 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1390/2749] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:26.724505 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724d34e-97aa-4634-9612-14f809004048 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:26.733918 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724d34e-97aa-4634-9612-14f809004048 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:26.739041 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e88aa5-aa3d-4dcf-93ad-5fce1f7de8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:26.739626 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e88aa5-aa3d-4dcf-93ad-5fce1f7de8c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:26.739904 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e88aa5-aa3d-4dcf-93ad-5fce1f7de8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:26.740176 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e88aa5-aa3d-4dcf-93ad-5fce1f7de8c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:26.740813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e88aa5-aa3d-4dcf-93ad-5fce1f7de8c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:20:26.741114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e88aa5-aa3d-4dcf-93ad-5fce1f7de8c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:26.742694 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e88aa5-aa3d-4dcf-93ad-5fce1f7de8c7 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:26.753985 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1360/2750] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 15:20:26.757971 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff25ef5-9935-47b1-bc07-63c8bfe33637 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:26.767386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff25ef5-9935-47b1-bc07-63c8bfe33637 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:26.772200 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c59cf-5721-4f0a-9ff0-ba38b10a2de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:26.772794 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c59cf-5721-4f0a-9ff0-ba38b10a2de2 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:26.773077 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c59cf-5721-4f0a-9ff0-ba38b10a2de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:26.773348 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c59cf-5721-4f0a-9ff0-ba38b10a2de2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:26.773969 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c59cf-5721-4f0a-9ff0-ba38b10a2de2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:26.775879 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c59cf-5721-4f0a-9ff0-ba38b10a2de2 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:26.937189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1391/2751] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:20:26 2022] POST /identity/v3/projects => generated 365 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:20:26.939714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e88aa5-aa3d-4dcf-93ad-5fce1f7de8c7 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:26.949745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e88aa5-aa3d-4dcf-93ad-5fce1f7de8c7 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:26.976443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:27.007039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c653d90-234a-4de7-93ff-36a2d584a929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:27.007760 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c653d90-234a-4de7-93ff-36a2d584a929 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:27.008387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c653d90-234a-4de7-93ff-36a2d584a929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:27.008729 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c653d90-234a-4de7-93ff-36a2d584a929 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:27.009328 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c653d90-234a-4de7-93ff-36a2d584a929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:27.011499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c653d90-234a-4de7-93ff-36a2d584a929 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:27.082443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1361/2752] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:20:26 2022] POST /identity/v3/users => generated 283 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:20:27.086535 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c59cf-5721-4f0a-9ff0-ba38b10a2de2 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:27.097645 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c59cf-5721-4f0a-9ff0-ba38b10a2de2 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:27.102230 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e192c7a-81da-47e4-be3a-b283f0ab10d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:27.102899 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e192c7a-81da-47e4-be3a-b283f0ab10d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:27.103179 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e192c7a-81da-47e4-be3a-b283f0ab10d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:27.103450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e192c7a-81da-47e4-be3a-b283f0ab10d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:27.104081 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e192c7a-81da-47e4-be3a-b283f0ab10d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:27.105664 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e192c7a-81da-47e4-be3a-b283f0ab10d9 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:27.115504 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1392/2753] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:27.119402 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c653d90-234a-4de7-93ff-36a2d584a929 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:27.129197 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c653d90-234a-4de7-93ff-36a2d584a929 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:27.133704 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14031b4a-51d1-4472-a782-6e27ee84152d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:27.134288 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14031b4a-51d1-4472-a782-6e27ee84152d admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:27.134566 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14031b4a-51d1-4472-a782-6e27ee84152d admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:27.134868 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14031b4a-51d1-4472-a782-6e27ee84152d admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:27.135498 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14031b4a-51d1-4472-a782-6e27ee84152d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:27.137324 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14031b4a-51d1-4472-a782-6e27ee84152d admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:27.147573 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1362/2754] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:27.151343 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e192c7a-81da-47e4-be3a-b283f0ab10d9 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:27.162079 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e192c7a-81da-47e4-be3a-b283f0ab10d9 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:27.166688 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4334bab-b5d0-4951-801b-ca6a79f0264c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:27.167360 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4334bab-b5d0-4951-801b-ca6a79f0264c admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:27.167642 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4334bab-b5d0-4951-801b-ca6a79f0264c admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:27.168047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4334bab-b5d0-4951-801b-ca6a79f0264c admin admin] PATH_INFO: `/v3/projects/e7aa0158c29a48cdae341c13a500977b/users/573905ef19b841f6aeb8c39d05bf8fd0/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:27.182311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4334bab-b5d0-4951-801b-ca6a79f0264c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7aa0158c29a48cdae341c13a500977b, user_id=573905ef19b841f6aeb8c39d05bf8fd0, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:27.185075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4334bab-b5d0-4951-801b-ca6a79f0264c admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:27.252528 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1393/2755] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:20:27 2022] PUT /identity/v3/projects/e7aa0158c29a48cdae341c13a500977b/users/573905ef19b841f6aeb8c39d05bf8fd0/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:27.264487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d10d4-4f7a-4138-952a-26aad6559dc8 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:27.264805 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d10d4-4f7a-4138-952a-26aad6559dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:27.265080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d10d4-4f7a-4138-952a-26aad6559dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:27.293312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-548d10d4-4f7a-4138-952a-26aad6559dc8 None None] MFA Rules not processed for user `573905ef19b841f6aeb8c39d05bf8fd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:27.294514 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-548d10d4-4f7a-4138-952a-26aad6559dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:27.356528 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1363/2756] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:27.361031 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4334bab-b5d0-4951-801b-ca6a79f0264c admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:27.421397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4334bab-b5d0-4951-801b-ca6a79f0264c admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:27.426198 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c53b83-2d30-409f-9562-571d2e1f151e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:27.426822 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c53b83-2d30-409f-9562-571d2e1f151e admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:27.427100 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c53b83-2d30-409f-9562-571d2e1f151e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:27.427370 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c53b83-2d30-409f-9562-571d2e1f151e admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:27.428027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c53b83-2d30-409f-9562-571d2e1f151e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:27.429892 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c53b83-2d30-409f-9562-571d2e1f151e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:27.505581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1394/2757] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:20:27 2022] POST /identity/v3/projects => generated 365 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:20:27.509453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d10d4-4f7a-4138-952a-26aad6559dc8 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:27.520895 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d10d4-4f7a-4138-952a-26aad6559dc8 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:27.575171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bdc37-d3ae-4d78-bdc1-38b3d534c84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:27.575763 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bdc37-d3ae-4d78-bdc1-38b3d534c84a admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:27.576042 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bdc37-d3ae-4d78-bdc1-38b3d534c84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:27.576313 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bdc37-d3ae-4d78-bdc1-38b3d534c84a admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:27.576925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bdc37-d3ae-4d78-bdc1-38b3d534c84a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:27.579449 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bdc37-d3ae-4d78-bdc1-38b3d534c84a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:27.641886 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1364/2758] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:20:27 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:20:27.645920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c53b83-2d30-409f-9562-571d2e1f151e admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:27.656519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c53b83-2d30-409f-9562-571d2e1f151e admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:27.661217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a86421-948c-48ea-bf6f-e44db33c7607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:27.661820 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a86421-948c-48ea-bf6f-e44db33c7607 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:27.662253 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a86421-948c-48ea-bf6f-e44db33c7607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:27.662540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a86421-948c-48ea-bf6f-e44db33c7607 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:27.663394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a86421-948c-48ea-bf6f-e44db33c7607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:27.664987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a86421-948c-48ea-bf6f-e44db33c7607 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:27.676012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1395/2759] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:27.680112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bdc37-d3ae-4d78-bdc1-38b3d534c84a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:27.690677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bdc37-d3ae-4d78-bdc1-38b3d534c84a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:27.695308 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf703bfe-2fb6-4b1f-8969-a61a02012556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:27.695949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf703bfe-2fb6-4b1f-8969-a61a02012556 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:27.696227 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf703bfe-2fb6-4b1f-8969-a61a02012556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:27.696500 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf703bfe-2fb6-4b1f-8969-a61a02012556 admin admin] PATH_INFO: `/v3/projects/f8bc9977f5a84e02b6d31c8388d331f7/users/71b20d5955c74823a58ae553cfdc45fe/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:27.710928 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf703bfe-2fb6-4b1f-8969-a61a02012556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8bc9977f5a84e02b6d31c8388d331f7, user_id=71b20d5955c74823a58ae553cfdc45fe, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:27.713649 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf703bfe-2fb6-4b1f-8969-a61a02012556 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:27.778704 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1365/2760] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:20:27 2022] PUT /identity/v3/projects/f8bc9977f5a84e02b6d31c8388d331f7/users/71b20d5955c74823a58ae553cfdc45fe/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:27.782622 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a86421-948c-48ea-bf6f-e44db33c7607 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:27.792800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a86421-948c-48ea-bf6f-e44db33c7607 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:27.813734 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:27.849409 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b1a19-17a6-480c-9624-52cd211941e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:27.850012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b1a19-17a6-480c-9624-52cd211941e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:27.850296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b1a19-17a6-480c-9624-52cd211941e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:27.850576 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b1a19-17a6-480c-9624-52cd211941e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:27.851339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b1a19-17a6-480c-9624-52cd211941e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:27.853062 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b1a19-17a6-480c-9624-52cd211941e0 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:27.863242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1396/2761] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:27.867554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf703bfe-2fb6-4b1f-8969-a61a02012556 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:27.872894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:27.878709 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf703bfe-2fb6-4b1f-8969-a61a02012556 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:27.883421 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ed206-eee2-4553-a433-eb7c46e0a976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:27.884044 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ed206-eee2-4553-a433-eb7c46e0a976 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:27.884325 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ed206-eee2-4553-a433-eb7c46e0a976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:27.884596 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ed206-eee2-4553-a433-eb7c46e0a976 admin admin] PATH_INFO: `/v3/projects/f8bc9977f5a84e02b6d31c8388d331f7/users/71b20d5955c74823a58ae553cfdc45fe/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:27.886813 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ed206-eee2-4553-a433-eb7c46e0a976 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8bc9977f5a84e02b6d31c8388d331f7, user_id=71b20d5955c74823a58ae553cfdc45fe, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:27.889430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ed206-eee2-4553-a433-eb7c46e0a976 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:27.899260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1366/2762] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:20:27 2022] PUT /identity/v3/projects/f8bc9977f5a84e02b6d31c8388d331f7/users/71b20d5955c74823a58ae553cfdc45fe/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:27.909157 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66f4a9-39b6-4ef7-9f89-8a84453d576f None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:27.909516 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66f4a9-39b6-4ef7-9f89-8a84453d576f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:27.909865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66f4a9-39b6-4ef7-9f89-8a84453d576f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:27.938653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-1f66f4a9-39b6-4ef7-9f89-8a84453d576f None None] MFA Rules not processed for user `71b20d5955c74823a58ae553cfdc45fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:27.939913 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-1f66f4a9-39b6-4ef7-9f89-8a84453d576f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:28.009021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1397/2763] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:27 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:28.020231 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ed206-eee2-4553-a433-eb7c46e0a976 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:28.021937 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a04ed206-eee2-4553-a433-eb7c46e0a976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:28.081759 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ed206-eee2-4553-a433-eb7c46e0a976 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:28.085467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ffe52-4d36-4fbe-bcfe-611fe8de3c47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:28.086079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ffe52-4d36-4fbe-bcfe-611fe8de3c47 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:28.086429 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ffe52-4d36-4fbe-bcfe-611fe8de3c47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:28.086755 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ffe52-4d36-4fbe-bcfe-611fe8de3c47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:28.100052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ffe52-4d36-4fbe-bcfe-611fe8de3c47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:28.100052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ffe52-4d36-4fbe-bcfe-611fe8de3c47 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:28.108200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1367/2764] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:20:28 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:20:28.319241 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66f4a9-39b6-4ef7-9f89-8a84453d576f None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:28.328984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66f4a9-39b6-4ef7-9f89-8a84453d576f None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:28.332525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821bdd9-de02-4303-accd-3851d3a26bb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:28.333117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821bdd9-de02-4303-accd-3851d3a26bb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:28.333394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821bdd9-de02-4303-accd-3851d3a26bb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:28.333662 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821bdd9-de02-4303-accd-3851d3a26bb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:28.388545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821bdd9-de02-4303-accd-3851d3a26bb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:28.389964 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821bdd9-de02-4303-accd-3851d3a26bb0 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:28.398738 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1398/2765] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:20:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:20:31.764706 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ffe52-4d36-4fbe-bcfe-611fe8de3c47 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:31.825344 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ffe52-4d36-4fbe-bcfe-611fe8de3c47 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:31.830363 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073276d8-b37c-4a38-81f8-9494d8ed1e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:31.831018 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073276d8-b37c-4a38-81f8-9494d8ed1e35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:31.831301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073276d8-b37c-4a38-81f8-9494d8ed1e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:31.831574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073276d8-b37c-4a38-81f8-9494d8ed1e35 admin admin] PATH_INFO: `/v3/users/573905ef19b841f6aeb8c39d05bf8fd0` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:31.832682 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073276d8-b37c-4a38-81f8-9494d8ed1e35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=573905ef19b841f6aeb8c39d05bf8fd0)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:31.834586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073276d8-b37c-4a38-81f8-9494d8ed1e35 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:32.008739 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-073276d8-b37c-4a38-81f8-9494d8ed1e35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '573905ef19b841f6aeb8c39d05bf8fd0'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:32.070185 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-073276d8-b37c-4a38-81f8-9494d8ed1e35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '573905ef19b841f6aeb8c39d05bf8fd0'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:32.070739 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-073276d8-b37c-4a38-81f8-9494d8ed1e35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '573905ef19b841f6aeb8c39d05bf8fd0'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:32.084244 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-073276d8-b37c-4a38-81f8-9494d8ed1e35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '573905ef19b841f6aeb8c39d05bf8fd0'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:32.091081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-073276d8-b37c-4a38-81f8-9494d8ed1e35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '573905ef19b841f6aeb8c39d05bf8fd0'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:32.098018 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1368/2766] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:20:31 2022] DELETE /identity/v3/users/573905ef19b841f6aeb8c39d05bf8fd0 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:32.102452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d795ca-2e33-47d5-a78e-0d9d8bb68e86 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:32.102922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d795ca-2e33-47d5-a78e-0d9d8bb68e86 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:32.103015 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d795ca-2e33-47d5-a78e-0d9d8bb68e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:32.122557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-c2d795ca-2e33-47d5-a78e-0d9d8bb68e86 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:32.123791 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c2d795ca-2e33-47d5-a78e-0d9d8bb68e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:32.163355 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:32.181312 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1399/2767] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:32.191507 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073276d8-b37c-4a38-81f8-9494d8ed1e35 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:32.192910 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-073276d8-b37c-4a38-81f8-9494d8ed1e35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:32.207454 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:32.250835 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073276d8-b37c-4a38-81f8-9494d8ed1e35 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:32.255638 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fdaec-e387-46f1-8007-80b547ef1506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:32.256251 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fdaec-e387-46f1-8007-80b547ef1506 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:32.256535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fdaec-e387-46f1-8007-80b547ef1506 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:32.256810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fdaec-e387-46f1-8007-80b547ef1506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:32.266263 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0fdaec-e387-46f1-8007-80b547ef1506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:32.268356 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0fdaec-e387-46f1-8007-80b547ef1506 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:32.277981 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1369/2768] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:20:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:20:32.695372 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d795ca-2e33-47d5-a78e-0d9d8bb68e86 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:32.699044 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c2d795ca-2e33-47d5-a78e-0d9d8bb68e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:32.709132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d795ca-2e33-47d5-a78e-0d9d8bb68e86 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:32.713784 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61570702-70f7-42e6-a5fd-78b098c44b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:32.716100 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61570702-70f7-42e6-a5fd-78b098c44b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:32.716100 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61570702-70f7-42e6-a5fd-78b098c44b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:32.716100 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61570702-70f7-42e6-a5fd-78b098c44b20 admin admin] PATH_INFO: `/v3/projects/e7aa0158c29a48cdae341c13a500977b` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:32.716100 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61570702-70f7-42e6-a5fd-78b098c44b20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7aa0158c29a48cdae341c13a500977b)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:32.718037 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61570702-70f7-42e6-a5fd-78b098c44b20 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:32.801923 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-61570702-70f7-42e6-a5fd-78b098c44b20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7aa0158c29a48cdae341c13a500977b', 'request_id': 'req-61570702-70f7-42e6-a5fd-78b098c44b20'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:32.808364 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-61570702-70f7-42e6-a5fd-78b098c44b20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7aa0158c29a48cdae341c13a500977b', 'request_id': 'req-61570702-70f7-42e6-a5fd-78b098c44b20'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:32.869726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-61570702-70f7-42e6-a5fd-78b098c44b20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7aa0158c29a48cdae341c13a500977b', 'request_id': 'req-61570702-70f7-42e6-a5fd-78b098c44b20'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:32.880918 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-61570702-70f7-42e6-a5fd-78b098c44b20 admin admin] The token cache is being invalidate because project e7aa0158c29a48cdae341c13a500977b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:20:32.881206 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-61570702-70f7-42e6-a5fd-78b098c44b20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:32.881859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-61570702-70f7-42e6-a5fd-78b098c44b20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:32.883381 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1400/2769] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:20:32 2022] DELETE /identity/v3/projects/e7aa0158c29a48cdae341c13a500977b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:32.887711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fdaec-e387-46f1-8007-80b547ef1506 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:32.897241 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fdaec-e387-46f1-8007-80b547ef1506 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:32.952853 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1001d1-0319-4d50-afae-5d579df0f3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:32.953431 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1001d1-0319-4d50-afae-5d579df0f3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:32.953712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1001d1-0319-4d50-afae-5d579df0f3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:32.953984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1001d1-0319-4d50-afae-5d579df0f3f8 admin admin] PATH_INFO: `/v3/users/71b20d5955c74823a58ae553cfdc45fe` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:32.955190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1001d1-0319-4d50-afae-5d579df0f3f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71b20d5955c74823a58ae553cfdc45fe)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:32.957123 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1001d1-0319-4d50-afae-5d579df0f3f8 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:33.101090 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-af1001d1-0319-4d50-afae-5d579df0f3f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71b20d5955c74823a58ae553cfdc45fe'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:33.155837 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-af1001d1-0319-4d50-afae-5d579df0f3f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71b20d5955c74823a58ae553cfdc45fe'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:33.156437 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-af1001d1-0319-4d50-afae-5d579df0f3f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71b20d5955c74823a58ae553cfdc45fe'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:33.169120 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-af1001d1-0319-4d50-afae-5d579df0f3f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71b20d5955c74823a58ae553cfdc45fe'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:33.176703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-af1001d1-0319-4d50-afae-5d579df0f3f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71b20d5955c74823a58ae553cfdc45fe'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:33.182534 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1370/2770] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:20:32 2022] DELETE /identity/v3/users/71b20d5955c74823a58ae553cfdc45fe => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:33.552043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61570702-70f7-42e6-a5fd-78b098c44b20 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:33.553339 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-61570702-70f7-42e6-a5fd-78b098c44b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:33.568635 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61570702-70f7-42e6-a5fd-78b098c44b20 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:33.628862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a31b1f-143a-4c1f-be8b-1de4eaad4ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:33.629495 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a31b1f-143a-4c1f-be8b-1de4eaad4ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:33.629773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a31b1f-143a-4c1f-be8b-1de4eaad4ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:33.630044 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a31b1f-143a-4c1f-be8b-1de4eaad4ac8 admin admin] PATH_INFO: `/v3/projects/f8bc9977f5a84e02b6d31c8388d331f7` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:33.631198 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a31b1f-143a-4c1f-be8b-1de4eaad4ac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8bc9977f5a84e02b6d31c8388d331f7)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:33.633141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a31b1f-143a-4c1f-be8b-1de4eaad4ac8 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:33.721658 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-79a31b1f-143a-4c1f-be8b-1de4eaad4ac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8bc9977f5a84e02b6d31c8388d331f7', 'request_id': 'req-79a31b1f-143a-4c1f-be8b-1de4eaad4ac8'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:33.727856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-79a31b1f-143a-4c1f-be8b-1de4eaad4ac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8bc9977f5a84e02b6d31c8388d331f7', 'request_id': 'req-79a31b1f-143a-4c1f-be8b-1de4eaad4ac8'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:33.772435 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-79a31b1f-143a-4c1f-be8b-1de4eaad4ac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8bc9977f5a84e02b6d31c8388d331f7', 'request_id': 'req-79a31b1f-143a-4c1f-be8b-1de4eaad4ac8'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:33.778661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:33.782943 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-79a31b1f-143a-4c1f-be8b-1de4eaad4ac8 admin admin] The token cache is being invalidate because project f8bc9977f5a84e02b6d31c8388d331f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:20:33.786581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-79a31b1f-143a-4c1f-be8b-1de4eaad4ac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:33.786581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-79a31b1f-143a-4c1f-be8b-1de4eaad4ac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:33.786581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1401/2771] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:20:33 2022] DELETE /identity/v3/projects/f8bc9977f5a84e02b6d31c8388d331f7 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:33.790128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1d3d6-ff6c-490d-a5ff-0aa77f3b0995 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:33.790419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1d3d6-ff6c-490d-a5ff-0aa77f3b0995 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:33.790697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1d3d6-ff6c-490d-a5ff-0aa77f3b0995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:33.811243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-33e1d3d6-ff6c-490d-a5ff-0aa77f3b0995 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:33.812475 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-33e1d3d6-ff6c-490d-a5ff-0aa77f3b0995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:33.842038 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:33.871282 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1371/2772] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:33.879723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057d061-b520-4748-853a-b16a1c638ca5 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:33.880008 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057d061-b520-4748-853a-b16a1c638ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:33.880278 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057d061-b520-4748-853a-b16a1c638ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:33.900935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-d057d061-b520-4748-853a-b16a1c638ca5 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:33.902158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d057d061-b520-4748-853a-b16a1c638ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:33.909946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1402/2773] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:33.914156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1d3d6-ff6c-490d-a5ff-0aa77f3b0995 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:33.924390 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1d3d6-ff6c-490d-a5ff-0aa77f3b0995 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:33.935472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5128c-3e7d-4343-82f2-93bd7e69cd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:33.935472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5128c-3e7d-4343-82f2-93bd7e69cd18 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:33.935472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5128c-3e7d-4343-82f2-93bd7e69cd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:33.935472 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5128c-3e7d-4343-82f2-93bd7e69cd18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:33.935848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5128c-3e7d-4343-82f2-93bd7e69cd18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:20:33.935848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5128c-3e7d-4343-82f2-93bd7e69cd18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:33.935848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5128c-3e7d-4343-82f2-93bd7e69cd18 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:33.944618 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1372/2774] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 15:20:33.948760 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057d061-b520-4748-853a-b16a1c638ca5 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:33.959021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057d061-b520-4748-853a-b16a1c638ca5 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:33.963693 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ad851-83b2-4ccd-98f5-1cbba780e7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:33.964288 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ad851-83b2-4ccd-98f5-1cbba780e7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:33.964572 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ad851-83b2-4ccd-98f5-1cbba780e7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:33.964848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ad851-83b2-4ccd-98f5-1cbba780e7b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:33.965516 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ad851-83b2-4ccd-98f5-1cbba780e7b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:33.967557 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ad851-83b2-4ccd-98f5-1cbba780e7b7 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:34.058977 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1403/2775] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:20:33 2022] POST /identity/v3/projects => generated 369 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:20:34.063186 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5128c-3e7d-4343-82f2-93bd7e69cd18 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:34.073796 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5128c-3e7d-4343-82f2-93bd7e69cd18 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:34.129776 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c0fa3-1e92-45fb-96a9-02157ed9df4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:34.130387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c0fa3-1e92-45fb-96a9-02157ed9df4b admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:34.130670 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c0fa3-1e92-45fb-96a9-02157ed9df4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:34.130987 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c0fa3-1e92-45fb-96a9-02157ed9df4b admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:34.131648 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c0fa3-1e92-45fb-96a9-02157ed9df4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:34.133632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c0fa3-1e92-45fb-96a9-02157ed9df4b admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:34.207401 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1373/2776] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:20:34 2022] POST /identity/v3/users => generated 285 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:20:34.210945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ad851-83b2-4ccd-98f5-1cbba780e7b7 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:34.221560 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ad851-83b2-4ccd-98f5-1cbba780e7b7 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:34.226296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fcfa2-f30a-42b7-a7b6-97daabbf2559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:34.226932 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fcfa2-f30a-42b7-a7b6-97daabbf2559 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:34.227219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fcfa2-f30a-42b7-a7b6-97daabbf2559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:34.227495 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fcfa2-f30a-42b7-a7b6-97daabbf2559 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:34.228133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fcfa2-f30a-42b7-a7b6-97daabbf2559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:34.229827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fcfa2-f30a-42b7-a7b6-97daabbf2559 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:34.240126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1404/2777] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:34.243894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c0fa3-1e92-45fb-96a9-02157ed9df4b admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:34.254483 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c0fa3-1e92-45fb-96a9-02157ed9df4b admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:34.259208 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31c766-ba2c-484c-926e-1b6ba09f15c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:34.259803 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31c766-ba2c-484c-926e-1b6ba09f15c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:34.260080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31c766-ba2c-484c-926e-1b6ba09f15c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:34.260351 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31c766-ba2c-484c-926e-1b6ba09f15c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:34.260980 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31c766-ba2c-484c-926e-1b6ba09f15c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:34.262829 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31c766-ba2c-484c-926e-1b6ba09f15c8 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:34.273177 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1374/2778] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:34.277431 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fcfa2-f30a-42b7-a7b6-97daabbf2559 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:34.287503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fcfa2-f30a-42b7-a7b6-97daabbf2559 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:34.292205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c97fd-b113-41a2-a9f7-cf0ee02bc2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:34.292850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c97fd-b113-41a2-a9f7-cf0ee02bc2eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:34.293134 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c97fd-b113-41a2-a9f7-cf0ee02bc2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:34.293888 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c97fd-b113-41a2-a9f7-cf0ee02bc2eb admin admin] PATH_INFO: `/v3/projects/0dedd8dc1d0d4bc392d8bb71fe97971a/users/286bc330ec594a7f8dfbe463b212bead/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:34.308357 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c97fd-b113-41a2-a9f7-cf0ee02bc2eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dedd8dc1d0d4bc392d8bb71fe97971a, user_id=286bc330ec594a7f8dfbe463b212bead, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:34.311181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c97fd-b113-41a2-a9f7-cf0ee02bc2eb admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:34.391015 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1405/2779] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:20:34 2022] PUT /identity/v3/projects/0dedd8dc1d0d4bc392d8bb71fe97971a/users/286bc330ec594a7f8dfbe463b212bead/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:34.402089 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b3fd6-eddd-4817-bff1-bcaaa92b44c9 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:34.402380 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b3fd6-eddd-4817-bff1-bcaaa92b44c9 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:34.403703 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b3fd6-eddd-4817-bff1-bcaaa92b44c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:34.430742 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-529b3fd6-eddd-4817-bff1-bcaaa92b44c9 None None] MFA Rules not processed for user `286bc330ec594a7f8dfbe463b212bead`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:34.432042 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-529b3fd6-eddd-4817-bff1-bcaaa92b44c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:34.493619 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1375/2780] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:34.498022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c97fd-b113-41a2-a9f7-cf0ee02bc2eb admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:34.558591 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c97fd-b113-41a2-a9f7-cf0ee02bc2eb admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:34.563458 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d9cef-8db3-4a1f-8e99-38564fb1ccde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:34.564141 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d9cef-8db3-4a1f-8e99-38564fb1ccde admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:34.564419 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d9cef-8db3-4a1f-8e99-38564fb1ccde admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:34.564683 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d9cef-8db3-4a1f-8e99-38564fb1ccde admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:34.565296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d9cef-8db3-4a1f-8e99-38564fb1ccde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:34.567198 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d9cef-8db3-4a1f-8e99-38564fb1ccde admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:34.643593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1406/2781] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:20:34 2022] POST /identity/v3/projects => generated 369 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:20:34.647653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b3fd6-eddd-4817-bff1-bcaaa92b44c9 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:34.658702 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b3fd6-eddd-4817-bff1-bcaaa92b44c9 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:34.684430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:34.713538 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fb064-ffba-4e58-957e-19d5796f427e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:34.714132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fb064-ffba-4e58-957e-19d5796f427e admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:34.714411 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fb064-ffba-4e58-957e-19d5796f427e admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:34.714679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fb064-ffba-4e58-957e-19d5796f427e admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:34.715316 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802fb064-ffba-4e58-957e-19d5796f427e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:34.717221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802fb064-ffba-4e58-957e-19d5796f427e admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:34.789856 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1376/2782] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:20:34 2022] POST /identity/v3/users => generated 291 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:20:34.793747 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d9cef-8db3-4a1f-8e99-38564fb1ccde admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:34.803997 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d9cef-8db3-4a1f-8e99-38564fb1ccde admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:34.808686 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f380194-019a-4169-84c7-dfd1f4e95f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:34.809274 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f380194-019a-4169-84c7-dfd1f4e95f7e admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:34.809551 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f380194-019a-4169-84c7-dfd1f4e95f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:34.809820 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f380194-019a-4169-84c7-dfd1f4e95f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:34.810515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f380194-019a-4169-84c7-dfd1f4e95f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:34.812124 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f380194-019a-4169-84c7-dfd1f4e95f7e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:34.822738 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1407/2783] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:34.826612 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fb064-ffba-4e58-957e-19d5796f427e admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:34.837303 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fb064-ffba-4e58-957e-19d5796f427e admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:34.841892 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dd12b-a673-49f6-9b13-f05665110921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:34.842530 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd12b-a673-49f6-9b13-f05665110921 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:34.842838 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd12b-a673-49f6-9b13-f05665110921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:34.843116 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd12b-a673-49f6-9b13-f05665110921 admin admin] PATH_INFO: `/v3/projects/d757101fbd174187a5ab6f23c0953557/users/08c2983a22b64926a22acc28f59461c7/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:34.857911 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dd12b-a673-49f6-9b13-f05665110921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d757101fbd174187a5ab6f23c0953557, user_id=08c2983a22b64926a22acc28f59461c7, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:34.860570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dd12b-a673-49f6-9b13-f05665110921 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:34.927397 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1377/2784] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:20:34 2022] PUT /identity/v3/projects/d757101fbd174187a5ab6f23c0953557/users/08c2983a22b64926a22acc28f59461c7/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:34.931052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f380194-019a-4169-84c7-dfd1f4e95f7e admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:34.940985 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f380194-019a-4169-84c7-dfd1f4e95f7e admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:34.996327 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797549ae-c48e-4084-9327-836c002a07a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:34.996923 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797549ae-c48e-4084-9327-836c002a07a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:34.997201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797549ae-c48e-4084-9327-836c002a07a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:34.997470 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797549ae-c48e-4084-9327-836c002a07a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:34.998096 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797549ae-c48e-4084-9327-836c002a07a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:34.999794 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797549ae-c48e-4084-9327-836c002a07a1 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:35.009336 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1408/2785] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:35.013708 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dd12b-a673-49f6-9b13-f05665110921 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:35.024519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dd12b-a673-49f6-9b13-f05665110921 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:35.029124 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3adbd-a559-439a-8128-c7265bc58167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:35.029770 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3adbd-a559-439a-8128-c7265bc58167 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:35.030108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3adbd-a559-439a-8128-c7265bc58167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:35.030387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3adbd-a559-439a-8128-c7265bc58167 admin admin] PATH_INFO: `/v3/projects/d757101fbd174187a5ab6f23c0953557/users/08c2983a22b64926a22acc28f59461c7/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:35.032346 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c3adbd-a559-439a-8128-c7265bc58167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d757101fbd174187a5ab6f23c0953557, user_id=08c2983a22b64926a22acc28f59461c7, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:35.035627 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c3adbd-a559-439a-8128-c7265bc58167 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:35.045567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1378/2786] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:20:35 2022] PUT /identity/v3/projects/d757101fbd174187a5ab6f23c0953557/users/08c2983a22b64926a22acc28f59461c7/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:20:35.055349 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59483d-63e4-4db6-b09f-a11b70065496 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:35.055639 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59483d-63e4-4db6-b09f-a11b70065496 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:35.055917 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59483d-63e4-4db6-b09f-a11b70065496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:35.063777 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:35.083560 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-aa59483d-63e4-4db6-b09f-a11b70065496 None None] MFA Rules not processed for user `08c2983a22b64926a22acc28f59461c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:35.084774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-aa59483d-63e4-4db6-b09f-a11b70065496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:35.154567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1409/2787] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:35.165284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3adbd-a559-439a-8128-c7265bc58167 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:35.178958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d3c3adbd-a559-439a-8128-c7265bc58167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:35.224091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3adbd-a559-439a-8128-c7265bc58167 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:35.227788 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ee68f-b209-4b32-9ee1-4129480d0763 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:35.228415 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ee68f-b209-4b32-9ee1-4129480d0763 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:35.228690 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ee68f-b209-4b32-9ee1-4129480d0763 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:35.228961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ee68f-b209-4b32-9ee1-4129480d0763 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:35.237925 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ee68f-b209-4b32-9ee1-4129480d0763 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:35.239391 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ee68f-b209-4b32-9ee1-4129480d0763 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:35.248787 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1379/2788] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:20:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:20:37.766228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59483d-63e4-4db6-b09f-a11b70065496 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:37.827842 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59483d-63e4-4db6-b09f-a11b70065496 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:37.833017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc57650-d044-4e99-b6f0-8ffb75ed8c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:37.833650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc57650-d044-4e99-b6f0-8ffb75ed8c01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:37.833933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc57650-d044-4e99-b6f0-8ffb75ed8c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:37.834208 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc57650-d044-4e99-b6f0-8ffb75ed8c01 admin admin] PATH_INFO: `/v3/users/286bc330ec594a7f8dfbe463b212bead` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:37.835344 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc57650-d044-4e99-b6f0-8ffb75ed8c01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=286bc330ec594a7f8dfbe463b212bead)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:37.837264 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc57650-d044-4e99-b6f0-8ffb75ed8c01 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:38.037877 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-fbc57650-d044-4e99-b6f0-8ffb75ed8c01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '286bc330ec594a7f8dfbe463b212bead'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:38.115754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-fbc57650-d044-4e99-b6f0-8ffb75ed8c01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '286bc330ec594a7f8dfbe463b212bead'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:38.128661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-fbc57650-d044-4e99-b6f0-8ffb75ed8c01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '286bc330ec594a7f8dfbe463b212bead'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:38.129286 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-fbc57650-d044-4e99-b6f0-8ffb75ed8c01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '286bc330ec594a7f8dfbe463b212bead'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:38.135981 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-fbc57650-d044-4e99-b6f0-8ffb75ed8c01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '286bc330ec594a7f8dfbe463b212bead'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:38.142233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1410/2789] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:20:37 2022] DELETE /identity/v3/users/286bc330ec594a7f8dfbe463b212bead => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:38.147455 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb70ab6-474f-431e-a9ef-8b8b40b36e4d None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:38.147904 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb70ab6-474f-431e-a9ef-8b8b40b36e4d None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:38.147992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb70ab6-474f-431e-a9ef-8b8b40b36e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:38.166825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-4bb70ab6-474f-431e-a9ef-8b8b40b36e4d None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:38.168029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-4bb70ab6-474f-431e-a9ef-8b8b40b36e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:38.224420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1380/2790] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:38.234003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc57650-d044-4e99-b6f0-8ffb75ed8c01 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:38.235417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-fbc57650-d044-4e99-b6f0-8ffb75ed8c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:38.246604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:38.294317 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc57650-d044-4e99-b6f0-8ffb75ed8c01 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:38.298297 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022a3e5-efb0-4268-a9d3-702a97250ac8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:38.298965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022a3e5-efb0-4268-a9d3-702a97250ac8 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:38.299268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022a3e5-efb0-4268-a9d3-702a97250ac8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:38.304835 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022a3e5-efb0-4268-a9d3-702a97250ac8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:38.309417 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a022a3e5-efb0-4268-a9d3-702a97250ac8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:38.310879 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a022a3e5-efb0-4268-a9d3-702a97250ac8 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:38.321849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1411/2791] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 15:20:38.700358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb70ab6-474f-431e-a9ef-8b8b40b36e4d None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:38.701937 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-4bb70ab6-474f-431e-a9ef-8b8b40b36e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:38.714450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb70ab6-474f-431e-a9ef-8b8b40b36e4d None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:38.719191 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35842dd4-00ec-4aea-ae17-be9d1e36b07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:38.719917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35842dd4-00ec-4aea-ae17-be9d1e36b07b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:38.720203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35842dd4-00ec-4aea-ae17-be9d1e36b07b admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:38.720480 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35842dd4-00ec-4aea-ae17-be9d1e36b07b admin admin] PATH_INFO: `/v3/projects/0dedd8dc1d0d4bc392d8bb71fe97971a` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:38.721708 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35842dd4-00ec-4aea-ae17-be9d1e36b07b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dedd8dc1d0d4bc392d8bb71fe97971a)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:38.723579 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35842dd4-00ec-4aea-ae17-be9d1e36b07b admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:38.746003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:38.831320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-35842dd4-00ec-4aea-ae17-be9d1e36b07b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dedd8dc1d0d4bc392d8bb71fe97971a', 'request_id': 'req-35842dd4-00ec-4aea-ae17-be9d1e36b07b'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:38.837246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-35842dd4-00ec-4aea-ae17-be9d1e36b07b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dedd8dc1d0d4bc392d8bb71fe97971a', 'request_id': 'req-35842dd4-00ec-4aea-ae17-be9d1e36b07b'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:38.881202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-35842dd4-00ec-4aea-ae17-be9d1e36b07b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dedd8dc1d0d4bc392d8bb71fe97971a', 'request_id': 'req-35842dd4-00ec-4aea-ae17-be9d1e36b07b'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:38.892350 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-35842dd4-00ec-4aea-ae17-be9d1e36b07b admin admin] The token cache is being invalidate because project 0dedd8dc1d0d4bc392d8bb71fe97971a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:20:38.892633 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-35842dd4-00ec-4aea-ae17-be9d1e36b07b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:38.893872 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-35842dd4-00ec-4aea-ae17-be9d1e36b07b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:38.895380 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1381/2792] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:20:38 2022] DELETE /identity/v3/projects/0dedd8dc1d0d4bc392d8bb71fe97971a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:38.899514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022a3e5-efb0-4268-a9d3-702a97250ac8 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:38.909672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022a3e5-efb0-4268-a9d3-702a97250ac8 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:38.966798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa80ed-5c0f-471a-bcf9-246bb7add26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:38.967453 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa80ed-5c0f-471a-bcf9-246bb7add26d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:38.967740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa80ed-5c0f-471a-bcf9-246bb7add26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:38.968014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa80ed-5c0f-471a-bcf9-246bb7add26d admin admin] PATH_INFO: `/v3/users/08c2983a22b64926a22acc28f59461c7` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:38.969137 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa80ed-5c0f-471a-bcf9-246bb7add26d admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c2983a22b64926a22acc28f59461c7)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:38.971123 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa80ed-5c0f-471a-bcf9-246bb7add26d admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:39.111821 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-3cfa80ed-5c0f-471a-bcf9-246bb7add26d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c2983a22b64926a22acc28f59461c7'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:39.155228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-3cfa80ed-5c0f-471a-bcf9-246bb7add26d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c2983a22b64926a22acc28f59461c7'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:39.167137 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-3cfa80ed-5c0f-471a-bcf9-246bb7add26d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c2983a22b64926a22acc28f59461c7'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:39.167764 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-3cfa80ed-5c0f-471a-bcf9-246bb7add26d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c2983a22b64926a22acc28f59461c7'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:39.174525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-3cfa80ed-5c0f-471a-bcf9-246bb7add26d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c2983a22b64926a22acc28f59461c7'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:39.180337 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1412/2793] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:20:38 2022] DELETE /identity/v3/users/08c2983a22b64926a22acc28f59461c7 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:39.545473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35842dd4-00ec-4aea-ae17-be9d1e36b07b admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:39.547098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-35842dd4-00ec-4aea-ae17-be9d1e36b07b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:39.560702 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35842dd4-00ec-4aea-ae17-be9d1e36b07b admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:39.617860 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed94d8-876c-4b51-88d5-fd4bcc69a614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:39.618481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed94d8-876c-4b51-88d5-fd4bcc69a614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:39.618759 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed94d8-876c-4b51-88d5-fd4bcc69a614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:39.619063 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed94d8-876c-4b51-88d5-fd4bcc69a614 admin admin] PATH_INFO: `/v3/projects/d757101fbd174187a5ab6f23c0953557` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:39.620178 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed94d8-876c-4b51-88d5-fd4bcc69a614 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d757101fbd174187a5ab6f23c0953557)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:39.622141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed94d8-876c-4b51-88d5-fd4bcc69a614 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:39.727996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2bed94d8-876c-4b51-88d5-fd4bcc69a614 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd757101fbd174187a5ab6f23c0953557', 'request_id': 'req-2bed94d8-876c-4b51-88d5-fd4bcc69a614'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:39.733354 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2bed94d8-876c-4b51-88d5-fd4bcc69a614 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd757101fbd174187a5ab6f23c0953557', 'request_id': 'req-2bed94d8-876c-4b51-88d5-fd4bcc69a614'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:39.789698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2bed94d8-876c-4b51-88d5-fd4bcc69a614 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd757101fbd174187a5ab6f23c0953557', 'request_id': 'req-2bed94d8-876c-4b51-88d5-fd4bcc69a614'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:39.801881 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2bed94d8-876c-4b51-88d5-fd4bcc69a614 admin admin] The token cache is being invalidate because project d757101fbd174187a5ab6f23c0953557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:20:39.802172 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2bed94d8-876c-4b51-88d5-fd4bcc69a614 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:39.802824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2bed94d8-876c-4b51-88d5-fd4bcc69a614 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:39.804309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1382/2794] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:20:39 2022] DELETE /identity/v3/projects/d757101fbd174187a5ab6f23c0953557 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:39.809427 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7efe9c-25e0-4a71-9e99-28ad90a35854 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:39.809711 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7efe9c-25e0-4a71-9e99-28ad90a35854 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:39.809983 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7efe9c-25e0-4a71-9e99-28ad90a35854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:39.829249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-7d7efe9c-25e0-4a71-9e99-28ad90a35854 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:39.830436 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7d7efe9c-25e0-4a71-9e99-28ad90a35854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:39.856105 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:39.889269 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1413/2795] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 15:20:39.898108 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2988a-1bfa-4445-92b3-27b82b7ef7f4 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:39.898393 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2988a-1bfa-4445-92b3-27b82b7ef7f4 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:39.898671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2988a-1bfa-4445-92b3-27b82b7ef7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:39.918567 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-e9f2988a-1bfa-4445-92b3-27b82b7ef7f4 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:39.919802 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e9f2988a-1bfa-4445-92b3-27b82b7ef7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:39.928324 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1383/2796] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:39.932433 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7efe9c-25e0-4a71-9e99-28ad90a35854 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:39.941965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7efe9c-25e0-4a71-9e99-28ad90a35854 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:39.946450 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c72b9-14db-475c-812d-458284022572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:39.947079 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c72b9-14db-475c-812d-458284022572 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:39.947357 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c72b9-14db-475c-812d-458284022572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:39.947627 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c72b9-14db-475c-812d-458284022572 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:39.948266 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c72b9-14db-475c-812d-458284022572 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:20:39.948565 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c72b9-14db-475c-812d-458284022572 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:39.950139 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c72b9-14db-475c-812d-458284022572 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:39.961492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1414/2797] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 15:20:39.965473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2988a-1bfa-4445-92b3-27b82b7ef7f4 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:39.975036 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2988a-1bfa-4445-92b3-27b82b7ef7f4 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:39.979890 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248ebf9-b13f-4266-8068-a6ecb517e977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:39.980455 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248ebf9-b13f-4266-8068-a6ecb517e977 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:39.980732 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248ebf9-b13f-4266-8068-a6ecb517e977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:39.981002 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248ebf9-b13f-4266-8068-a6ecb517e977 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:39.981623 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248ebf9-b13f-4266-8068-a6ecb517e977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:39.983611 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248ebf9-b13f-4266-8068-a6ecb517e977 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:40.089083 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1384/2798] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:20:39 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:20:40.093280 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c72b9-14db-475c-812d-458284022572 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:40.104037 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c72b9-14db-475c-812d-458284022572 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:40.159712 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386356a6-ffe1-4a75-a12d-8cad12a99973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:40.160236 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386356a6-ffe1-4a75-a12d-8cad12a99973 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:40.160517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386356a6-ffe1-4a75-a12d-8cad12a99973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:40.160792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386356a6-ffe1-4a75-a12d-8cad12a99973 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:40.161401 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386356a6-ffe1-4a75-a12d-8cad12a99973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:40.163381 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386356a6-ffe1-4a75-a12d-8cad12a99973 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:40.224901 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1415/2799] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:20:40 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:20:40.229192 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248ebf9-b13f-4266-8068-a6ecb517e977 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:40.239546 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248ebf9-b13f-4266-8068-a6ecb517e977 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:40.244182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb956ca-f1bd-447f-aa3f-a1c35e9c7e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:40.248542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb956ca-f1bd-447f-aa3f-a1c35e9c7e3a admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:40.248542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb956ca-f1bd-447f-aa3f-a1c35e9c7e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:40.248542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb956ca-f1bd-447f-aa3f-a1c35e9c7e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:40.248542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb956ca-f1bd-447f-aa3f-a1c35e9c7e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:40.248542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb956ca-f1bd-447f-aa3f-a1c35e9c7e3a admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:40.249594 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:40.257811 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1385/2800] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:40.261958 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386356a6-ffe1-4a75-a12d-8cad12a99973 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:40.272522 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386356a6-ffe1-4a75-a12d-8cad12a99973 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:40.277267 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34112955-2088-4e2e-a629-b92f2a360d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:40.277850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34112955-2088-4e2e-a629-b92f2a360d06 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:40.278132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34112955-2088-4e2e-a629-b92f2a360d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:40.278410 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34112955-2088-4e2e-a629-b92f2a360d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:40.279095 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34112955-2088-4e2e-a629-b92f2a360d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:40.280766 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34112955-2088-4e2e-a629-b92f2a360d06 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:40.292043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1416/2801] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:40.296047 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb956ca-f1bd-447f-aa3f-a1c35e9c7e3a admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:40.306442 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb956ca-f1bd-447f-aa3f-a1c35e9c7e3a admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:40.311280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157fdc15-1e0d-4df5-9c9c-9902d5629c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:40.311932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fdc15-1e0d-4df5-9c9c-9902d5629c0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:40.312211 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fdc15-1e0d-4df5-9c9c-9902d5629c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:40.312482 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fdc15-1e0d-4df5-9c9c-9902d5629c0b admin admin] PATH_INFO: `/v3/projects/7941c89bf9ec4c508839b3c7dd464f43/users/31d56ba0c76a43d791ce529260617251/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:40.327286 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157fdc15-1e0d-4df5-9c9c-9902d5629c0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7941c89bf9ec4c508839b3c7dd464f43, user_id=31d56ba0c76a43d791ce529260617251, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:40.329955 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157fdc15-1e0d-4df5-9c9c-9902d5629c0b admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:40.406662 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1386/2802] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:20:40 2022] PUT /identity/v3/projects/7941c89bf9ec4c508839b3c7dd464f43/users/31d56ba0c76a43d791ce529260617251/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:20:40.416974 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a520b35-1bf1-481d-b44a-feba9ecb2a97 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:40.417197 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a520b35-1bf1-481d-b44a-feba9ecb2a97 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:40.417475 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a520b35-1bf1-481d-b44a-feba9ecb2a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:40.445302 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-8a520b35-1bf1-481d-b44a-feba9ecb2a97 None None] MFA Rules not processed for user `31d56ba0c76a43d791ce529260617251`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:40.446494 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8a520b35-1bf1-481d-b44a-feba9ecb2a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:40.507914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1417/2803] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:40.512722 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157fdc15-1e0d-4df5-9c9c-9902d5629c0b admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:40.572254 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157fdc15-1e0d-4df5-9c9c-9902d5629c0b admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:40.576875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8e123-bca1-48c0-8b13-230b855481fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:40.577470 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8e123-bca1-48c0-8b13-230b855481fa admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:40.577745 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8e123-bca1-48c0-8b13-230b855481fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:40.578016 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8e123-bca1-48c0-8b13-230b855481fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:40.578636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8e123-bca1-48c0-8b13-230b855481fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:40.581163 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8e123-bca1-48c0-8b13-230b855481fa admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:40.648764 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1387/2804] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:20:40 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:20:40.652844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a520b35-1bf1-481d-b44a-feba9ecb2a97 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:40.663477 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a520b35-1bf1-481d-b44a-feba9ecb2a97 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:40.688058 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:40.722004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb8d68f-d29a-4c6f-9c8c-9f4644db6df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:40.722618 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8d68f-d29a-4c6f-9c8c-9f4644db6df9 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:40.722936 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8d68f-d29a-4c6f-9c8c-9f4644db6df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:40.723215 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8d68f-d29a-4c6f-9c8c-9f4644db6df9 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:40.723866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb8d68f-d29a-4c6f-9c8c-9f4644db6df9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:40.725876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb8d68f-d29a-4c6f-9c8c-9f4644db6df9 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:40.778538 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1418/2805] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:20:40 2022] POST /identity/v3/users => generated 298 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:20:40.782361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8e123-bca1-48c0-8b13-230b855481fa admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:40.793052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8e123-bca1-48c0-8b13-230b855481fa admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:40.797907 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533d7f0-5578-48e0-a82c-be827c0c042d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:40.798542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533d7f0-5578-48e0-a82c-be827c0c042d admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:40.798853 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533d7f0-5578-48e0-a82c-be827c0c042d admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:40.799131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533d7f0-5578-48e0-a82c-be827c0c042d admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:40.800336 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533d7f0-5578-48e0-a82c-be827c0c042d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:40.801924 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533d7f0-5578-48e0-a82c-be827c0c042d admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:40.812454 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1388/2806] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:40.817069 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb8d68f-d29a-4c6f-9c8c-9f4644db6df9 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:40.826899 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb8d68f-d29a-4c6f-9c8c-9f4644db6df9 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:40.831596 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5648b8b-7180-482a-87fa-f10ab3f32c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:40.832234 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5648b8b-7180-482a-87fa-f10ab3f32c37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:40.832511 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5648b8b-7180-482a-87fa-f10ab3f32c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:40.832781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5648b8b-7180-482a-87fa-f10ab3f32c37 admin admin] PATH_INFO: `/v3/projects/6c419d5959554b2487f159a495135c19/users/1eab5c0c4dc04d31be814232da602a67/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:40.846979 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5648b8b-7180-482a-87fa-f10ab3f32c37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c419d5959554b2487f159a495135c19, user_id=1eab5c0c4dc04d31be814232da602a67, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:40.849583 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5648b8b-7180-482a-87fa-f10ab3f32c37 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:40.903774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1419/2807] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:20:40 2022] PUT /identity/v3/projects/6c419d5959554b2487f159a495135c19/users/1eab5c0c4dc04d31be814232da602a67/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:40.907568 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533d7f0-5578-48e0-a82c-be827c0c042d admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:40.917736 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533d7f0-5578-48e0-a82c-be827c0c042d admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:40.972697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476cf64-90a9-4a84-a901-c5a722ffafa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:40.973298 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476cf64-90a9-4a84-a901-c5a722ffafa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:40.973583 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476cf64-90a9-4a84-a901-c5a722ffafa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:40.973859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476cf64-90a9-4a84-a901-c5a722ffafa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:40.974491 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476cf64-90a9-4a84-a901-c5a722ffafa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:40.976115 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476cf64-90a9-4a84-a901-c5a722ffafa5 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:40.986052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1389/2808] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:40.990600 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5648b8b-7180-482a-87fa-f10ab3f32c37 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:41.000850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5648b8b-7180-482a-87fa-f10ab3f32c37 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:41.006055 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba7990-0bb8-4936-9c6b-93748e7f5e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:41.006697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba7990-0bb8-4936-9c6b-93748e7f5e4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:41.007082 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba7990-0bb8-4936-9c6b-93748e7f5e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:41.007362 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba7990-0bb8-4936-9c6b-93748e7f5e4d admin admin] PATH_INFO: `/v3/projects/6c419d5959554b2487f159a495135c19/users/1eab5c0c4dc04d31be814232da602a67/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:41.009289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba7990-0bb8-4936-9c6b-93748e7f5e4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c419d5959554b2487f159a495135c19, user_id=1eab5c0c4dc04d31be814232da602a67, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:41.012224 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba7990-0bb8-4936-9c6b-93748e7f5e4d admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:41.022190 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1420/2809] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:20:40 2022] PUT /identity/v3/projects/6c419d5959554b2487f159a495135c19/users/1eab5c0c4dc04d31be814232da602a67/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:41.032109 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4642f-8d87-430f-b7a9-8bf6134125ef None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:41.032397 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4642f-8d87-430f-b7a9-8bf6134125ef None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:41.032706 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4642f-8d87-430f-b7a9-8bf6134125ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:41.059972 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-02f4642f-8d87-430f-b7a9-8bf6134125ef None None] MFA Rules not processed for user `1eab5c0c4dc04d31be814232da602a67`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:41.061443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-02f4642f-8d87-430f-b7a9-8bf6134125ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:41.093635 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:41.129136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1390/2810] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 15:20:41.140783 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba7990-0bb8-4936-9c6b-93748e7f5e4d admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:41.142516 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-58ba7990-0bb8-4936-9c6b-93748e7f5e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:41.199075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba7990-0bb8-4936-9c6b-93748e7f5e4d admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:41.202620 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954aff4-ffa0-442e-a4a5-4cd7c1815cf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:41.203236 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954aff4-ffa0-442e-a4a5-4cd7c1815cf7 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:41.203514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954aff4-ffa0-442e-a4a5-4cd7c1815cf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:41.203784 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954aff4-ffa0-442e-a4a5-4cd7c1815cf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:41.212416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954aff4-ffa0-442e-a4a5-4cd7c1815cf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:41.213824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954aff4-ffa0-442e-a4a5-4cd7c1815cf7 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:41.223587 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1421/2811] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:20:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:20:41.353018 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4642f-8d87-430f-b7a9-8bf6134125ef None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:41.363111 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4642f-8d87-430f-b7a9-8bf6134125ef None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:41.366782 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f9b91-5ef3-4d63-8b91-7962c808593f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:41.367393 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f9b91-5ef3-4d63-8b91-7962c808593f service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:41.367677 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f9b91-5ef3-4d63-8b91-7962c808593f service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:41.367951 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f9b91-5ef3-4d63-8b91-7962c808593f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:41.425068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f9b91-5ef3-4d63-8b91-7962c808593f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:41.426489 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f9b91-5ef3-4d63-8b91-7962c808593f service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:41.435129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1391/2812] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:20:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:20:41.952369 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954aff4-ffa0-442e-a4a5-4cd7c1815cf7 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:42.015052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954aff4-ffa0-442e-a4a5-4cd7c1815cf7 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:42.020129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afdd0e-c96a-4343-a504-9df73b064e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:42.020776 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afdd0e-c96a-4343-a504-9df73b064e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:42.021054 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afdd0e-c96a-4343-a504-9df73b064e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:42.021326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afdd0e-c96a-4343-a504-9df73b064e6a admin admin] PATH_INFO: `/v3/users/31d56ba0c76a43d791ce529260617251` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:42.022493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afdd0e-c96a-4343-a504-9df73b064e6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=31d56ba0c76a43d791ce529260617251)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:42.024484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afdd0e-c96a-4343-a504-9df73b064e6a admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:42.046617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:42.163134 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-79afdd0e-c96a-4343-a504-9df73b064e6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31d56ba0c76a43d791ce529260617251'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:42.212505 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-79afdd0e-c96a-4343-a504-9df73b064e6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31d56ba0c76a43d791ce529260617251'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:42.225023 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-79afdd0e-c96a-4343-a504-9df73b064e6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31d56ba0c76a43d791ce529260617251'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:42.225710 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-79afdd0e-c96a-4343-a504-9df73b064e6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31d56ba0c76a43d791ce529260617251'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:42.232824 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-79afdd0e-c96a-4343-a504-9df73b064e6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31d56ba0c76a43d791ce529260617251'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:42.239201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1422/2813] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:20:41 2022] DELETE /identity/v3/users/31d56ba0c76a43d791ce529260617251 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:42.243901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2edb3-7e45-4124-a43f-ed0e4ff36273 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:42.244341 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2edb3-7e45-4124-a43f-ed0e4ff36273 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:42.244341 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2edb3-7e45-4124-a43f-ed0e4ff36273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:42.263639 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-51f2edb3-7e45-4124-a43f-ed0e4ff36273 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:42.264828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-51f2edb3-7e45-4124-a43f-ed0e4ff36273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:42.323516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1392/2814] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:42.332993 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afdd0e-c96a-4343-a504-9df73b064e6a admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:42.334406 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-79afdd0e-c96a-4343-a504-9df73b064e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:42.391005 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afdd0e-c96a-4343-a504-9df73b064e6a admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:42.394591 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fdad58-3c51-47f8-8e0b-58fbf951825a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:42.395214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdad58-3c51-47f8-8e0b-58fbf951825a service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:42.395492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdad58-3c51-47f8-8e0b-58fbf951825a service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:42.395762 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdad58-3c51-47f8-8e0b-58fbf951825a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:42.404550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fdad58-3c51-47f8-8e0b-58fbf951825a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:42.405984 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fdad58-3c51-47f8-8e0b-58fbf951825a service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:42.415197 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1423/2815] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:20:42.803771 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2edb3-7e45-4124-a43f-ed0e4ff36273 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:42.805283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-51f2edb3-7e45-4124-a43f-ed0e4ff36273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:42.820598 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2edb3-7e45-4124-a43f-ed0e4ff36273 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:42.825941 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36636580-7815-4d21-916c-cb5923504f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:42.826695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36636580-7815-4d21-916c-cb5923504f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:42.827097 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36636580-7815-4d21-916c-cb5923504f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:42.827473 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36636580-7815-4d21-916c-cb5923504f0b admin admin] PATH_INFO: `/v3/projects/7941c89bf9ec4c508839b3c7dd464f43` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:42.828743 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36636580-7815-4d21-916c-cb5923504f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7941c89bf9ec4c508839b3c7dd464f43)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:42.831089 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36636580-7815-4d21-916c-cb5923504f0b admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:42.938338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-36636580-7815-4d21-916c-cb5923504f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7941c89bf9ec4c508839b3c7dd464f43', 'request_id': 'req-36636580-7815-4d21-916c-cb5923504f0b'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:42.943918 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-36636580-7815-4d21-916c-cb5923504f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7941c89bf9ec4c508839b3c7dd464f43', 'request_id': 'req-36636580-7815-4d21-916c-cb5923504f0b'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:42.947453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:43.029777 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-36636580-7815-4d21-916c-cb5923504f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7941c89bf9ec4c508839b3c7dd464f43', 'request_id': 'req-36636580-7815-4d21-916c-cb5923504f0b'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:43.041829 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-36636580-7815-4d21-916c-cb5923504f0b admin admin] The token cache is being invalidate because project 7941c89bf9ec4c508839b3c7dd464f43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:20:43.042550 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-36636580-7815-4d21-916c-cb5923504f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:43.043157 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-36636580-7815-4d21-916c-cb5923504f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:43.044665 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1393/2816] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:20:42 2022] DELETE /identity/v3/projects/7941c89bf9ec4c508839b3c7dd464f43 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:43.048650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fdad58-3c51-47f8-8e0b-58fbf951825a service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:43.058457 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fdad58-3c51-47f8-8e0b-58fbf951825a service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:43.114850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00851f3-973f-446e-a68c-d70b76c2c524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:43.115416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00851f3-973f-446e-a68c-d70b76c2c524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:43.115695 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00851f3-973f-446e-a68c-d70b76c2c524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:43.115970 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00851f3-973f-446e-a68c-d70b76c2c524 admin admin] PATH_INFO: `/v3/users/1eab5c0c4dc04d31be814232da602a67` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:43.117060 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00851f3-973f-446e-a68c-d70b76c2c524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eab5c0c4dc04d31be814232da602a67)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:43.119039 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00851f3-973f-446e-a68c-d70b76c2c524 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:43.230775 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:43.232332 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f00851f3-973f-446e-a68c-d70b76c2c524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eab5c0c4dc04d31be814232da602a67'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:43.281018 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f00851f3-973f-446e-a68c-d70b76c2c524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eab5c0c4dc04d31be814232da602a67'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:43.294265 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f00851f3-973f-446e-a68c-d70b76c2c524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eab5c0c4dc04d31be814232da602a67'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:43.294914 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f00851f3-973f-446e-a68c-d70b76c2c524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eab5c0c4dc04d31be814232da602a67'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:43.301654 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-f00851f3-973f-446e-a68c-d70b76c2c524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eab5c0c4dc04d31be814232da602a67'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:43.307681 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1424/2817] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:20:43 2022] DELETE /identity/v3/users/1eab5c0c4dc04d31be814232da602a67 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:43.717112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36636580-7815-4d21-916c-cb5923504f0b admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:43.718626 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-36636580-7815-4d21-916c-cb5923504f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:43.731917 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36636580-7815-4d21-916c-cb5923504f0b admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:43.789536 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68fb2c-c2d2-4ac5-85fe-d4477a32db63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:43.790203 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68fb2c-c2d2-4ac5-85fe-d4477a32db63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:43.790485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68fb2c-c2d2-4ac5-85fe-d4477a32db63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:43.790762 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68fb2c-c2d2-4ac5-85fe-d4477a32db63 admin admin] PATH_INFO: `/v3/projects/6c419d5959554b2487f159a495135c19` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:43.791908 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68fb2c-c2d2-4ac5-85fe-d4477a32db63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c419d5959554b2487f159a495135c19)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:43.793816 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68fb2c-c2d2-4ac5-85fe-d4477a32db63 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:43.887988 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3b68fb2c-c2d2-4ac5-85fe-d4477a32db63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c419d5959554b2487f159a495135c19', 'request_id': 'req-3b68fb2c-c2d2-4ac5-85fe-d4477a32db63'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:43.894417 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3b68fb2c-c2d2-4ac5-85fe-d4477a32db63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c419d5959554b2487f159a495135c19', 'request_id': 'req-3b68fb2c-c2d2-4ac5-85fe-d4477a32db63'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:43.944600 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3b68fb2c-c2d2-4ac5-85fe-d4477a32db63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c419d5959554b2487f159a495135c19', 'request_id': 'req-3b68fb2c-c2d2-4ac5-85fe-d4477a32db63'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:43.956054 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3b68fb2c-c2d2-4ac5-85fe-d4477a32db63 admin admin] The token cache is being invalidate because project 6c419d5959554b2487f159a495135c19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:20:43.956346 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3b68fb2c-c2d2-4ac5-85fe-d4477a32db63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:43.957043 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-3b68fb2c-c2d2-4ac5-85fe-d4477a32db63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:43.959117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1394/2818] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:20:43 2022] DELETE /identity/v3/projects/6c419d5959554b2487f159a495135c19 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:43.964896 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29378f-fe68-4b1b-98d7-b8f4af37f8e5 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:43.965126 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29378f-fe68-4b1b-98d7-b8f4af37f8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:43.965402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29378f-fe68-4b1b-98d7-b8f4af37f8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:43.985197 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-1c29378f-fe68-4b1b-98d7-b8f4af37f8e5 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:43.986426 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-1c29378f-fe68-4b1b-98d7-b8f4af37f8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:44.044012 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1425/2819] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:44.051865 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fef70-b2bd-4365-b8c0-74e00ba8cf02 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:44.052150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fef70-b2bd-4365-b8c0-74e00ba8cf02 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:44.052456 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fef70-b2bd-4365-b8c0-74e00ba8cf02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:44.072077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-a37fef70-b2bd-4365-b8c0-74e00ba8cf02 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:44.073292 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a37fef70-b2bd-4365-b8c0-74e00ba8cf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:44.081051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1395/2820] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:44.086015 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29378f-fe68-4b1b-98d7-b8f4af37f8e5 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:44.095862 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29378f-fe68-4b1b-98d7-b8f4af37f8e5 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:44.100510 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b54d00-f510-4d00-808b-5f63a5e7d326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:44.101111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b54d00-f510-4d00-808b-5f63a5e7d326 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:44.101393 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b54d00-f510-4d00-808b-5f63a5e7d326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:44.101669 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b54d00-f510-4d00-808b-5f63a5e7d326 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:44.102313 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b54d00-f510-4d00-808b-5f63a5e7d326 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:20:44.102615 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b54d00-f510-4d00-808b-5f63a5e7d326 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:44.104279 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b54d00-f510-4d00-808b-5f63a5e7d326 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:44.115809 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1426/2821] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 15:20:44.120159 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fef70-b2bd-4365-b8c0-74e00ba8cf02 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:44.129749 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fef70-b2bd-4365-b8c0-74e00ba8cf02 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:44.134314 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c23939-fd24-42ac-9c96-d9d11022e544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:44.134933 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c23939-fd24-42ac-9c96-d9d11022e544 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:44.135213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c23939-fd24-42ac-9c96-d9d11022e544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:44.135484 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c23939-fd24-42ac-9c96-d9d11022e544 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:44.136107 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c23939-fd24-42ac-9c96-d9d11022e544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:44.138244 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c23939-fd24-42ac-9c96-d9d11022e544 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:44.220047 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1396/2822] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:20:44 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:20:44.224233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b54d00-f510-4d00-808b-5f63a5e7d326 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:44.234209 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b54d00-f510-4d00-808b-5f63a5e7d326 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:44.292294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3f648-bbcc-4004-acb4-e6820d63e748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:44.292908 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f648-bbcc-4004-acb4-e6820d63e748 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:44.293194 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f648-bbcc-4004-acb4-e6820d63e748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:44.293474 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f648-bbcc-4004-acb4-e6820d63e748 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:44.294143 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3f648-bbcc-4004-acb4-e6820d63e748 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:44.296163 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3f648-bbcc-4004-acb4-e6820d63e748 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:44.418538 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1427/2823] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:20:44 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:20:44.422686 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c23939-fd24-42ac-9c96-d9d11022e544 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:44.432857 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c23939-fd24-42ac-9c96-d9d11022e544 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:44.437396 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ad936-4e45-4fcb-889e-5f550c012fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:44.439683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ad936-4e45-4fcb-889e-5f550c012fad admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:44.439683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ad936-4e45-4fcb-889e-5f550c012fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:44.439683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ad936-4e45-4fcb-889e-5f550c012fad admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:44.439683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ad936-4e45-4fcb-889e-5f550c012fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:44.440786 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ad936-4e45-4fcb-889e-5f550c012fad admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:44.450936 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1397/2824] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:44.454765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3f648-bbcc-4004-acb4-e6820d63e748 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:44.465201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3f648-bbcc-4004-acb4-e6820d63e748 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:44.469806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02831af1-b5a2-48fa-a363-4b52930da6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:44.470447 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02831af1-b5a2-48fa-a363-4b52930da6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:44.470680 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02831af1-b5a2-48fa-a363-4b52930da6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:44.470986 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02831af1-b5a2-48fa-a363-4b52930da6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:44.471623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02831af1-b5a2-48fa-a363-4b52930da6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:44.473195 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02831af1-b5a2-48fa-a363-4b52930da6a3 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:44.484551 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1428/2825] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:44.488954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ad936-4e45-4fcb-889e-5f550c012fad admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:44.498720 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ad936-4e45-4fcb-889e-5f550c012fad admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:44.503219 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6cf5c-d933-404a-85ac-aaeeca02d6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:44.503854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6cf5c-d933-404a-85ac-aaeeca02d6c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:44.504688 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6cf5c-d933-404a-85ac-aaeeca02d6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:44.504964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6cf5c-d933-404a-85ac-aaeeca02d6c3 admin admin] PATH_INFO: `/v3/projects/1ebfb382b98d42f18e57a6c310146672/users/978b3d73cdde49b08913405cc3065c3d/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:44.519312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6cf5c-d933-404a-85ac-aaeeca02d6c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ebfb382b98d42f18e57a6c310146672, user_id=978b3d73cdde49b08913405cc3065c3d, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:44.521948 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6cf5c-d933-404a-85ac-aaeeca02d6c3 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:44.624803 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1398/2826] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:20:44 2022] PUT /identity/v3/projects/1ebfb382b98d42f18e57a6c310146672/users/978b3d73cdde49b08913405cc3065c3d/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:44.635455 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef114c-9203-465c-a355-10aacacb280b None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:44.635739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef114c-9203-465c-a355-10aacacb280b None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:44.636013 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef114c-9203-465c-a355-10aacacb280b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:44.662775 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-a0ef114c-9203-465c-a355-10aacacb280b None None] MFA Rules not processed for user `978b3d73cdde49b08913405cc3065c3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:44.664003 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-a0ef114c-9203-465c-a355-10aacacb280b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:44.680826 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:44.727321 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1429/2827] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:44.731710 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6cf5c-d933-404a-85ac-aaeeca02d6c3 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:44.751259 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:44.793382 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6cf5c-d933-404a-85ac-aaeeca02d6c3 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:44.798137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff247f9-43ab-46ad-bcd5-18226dd551a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:44.798740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff247f9-43ab-46ad-bcd5-18226dd551a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:44.799053 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff247f9-43ab-46ad-bcd5-18226dd551a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:44.799327 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff247f9-43ab-46ad-bcd5-18226dd551a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:44.799945 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff247f9-43ab-46ad-bcd5-18226dd551a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:44.802354 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff247f9-43ab-46ad-bcd5-18226dd551a0 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:44.882312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1399/2828] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:20:44 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:20:44.886636 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef114c-9203-465c-a355-10aacacb280b None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:44.897044 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef114c-9203-465c-a355-10aacacb280b None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:44.952683 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492f2ce-f4f5-49db-834b-947796cb91de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:44.953404 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492f2ce-f4f5-49db-834b-947796cb91de admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:44.953688 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492f2ce-f4f5-49db-834b-947796cb91de admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:44.953966 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492f2ce-f4f5-49db-834b-947796cb91de admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:44.954632 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492f2ce-f4f5-49db-834b-947796cb91de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:44.956608 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492f2ce-f4f5-49db-834b-947796cb91de admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:45.030028 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1430/2829] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:20:44 2022] POST /identity/v3/users => generated 298 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:20:45.033997 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff247f9-43ab-46ad-bcd5-18226dd551a0 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:45.045155 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff247f9-43ab-46ad-bcd5-18226dd551a0 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:45.049674 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a81461-b996-4fcf-8132-0901f00a3364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:45.050270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a81461-b996-4fcf-8132-0901f00a3364 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:45.050553 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a81461-b996-4fcf-8132-0901f00a3364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:45.051486 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a81461-b996-4fcf-8132-0901f00a3364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:45.052129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a81461-b996-4fcf-8132-0901f00a3364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:45.053725 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a81461-b996-4fcf-8132-0901f00a3364 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:45.064309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1400/2830] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:45.068227 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492f2ce-f4f5-49db-834b-947796cb91de admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:45.078425 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492f2ce-f4f5-49db-834b-947796cb91de admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:45.083101 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96df41-c30f-4966-a84e-a42bf0c50db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:45.083749 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96df41-c30f-4966-a84e-a42bf0c50db3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:45.084032 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96df41-c30f-4966-a84e-a42bf0c50db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:45.084308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96df41-c30f-4966-a84e-a42bf0c50db3 admin admin] PATH_INFO: `/v3/projects/624d542e011149a7b428605c4405d833/users/be43d129d35b4be281e849ce89912a61/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:45.099600 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96df41-c30f-4966-a84e-a42bf0c50db3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=624d542e011149a7b428605c4405d833, user_id=be43d129d35b4be281e849ce89912a61, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:45.102252 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96df41-c30f-4966-a84e-a42bf0c50db3 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:45.167072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1431/2831] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:20:45 2022] PUT /identity/v3/projects/624d542e011149a7b428605c4405d833/users/be43d129d35b4be281e849ce89912a61/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:45.171047 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a81461-b996-4fcf-8132-0901f00a3364 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:45.181524 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a81461-b996-4fcf-8132-0901f00a3364 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:45.236391 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7941bfd2-d2c6-461e-82f8-285d5d36beb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:45.236932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941bfd2-d2c6-461e-82f8-285d5d36beb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:45.237219 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941bfd2-d2c6-461e-82f8-285d5d36beb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:45.237499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941bfd2-d2c6-461e-82f8-285d5d36beb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:45.238136 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7941bfd2-d2c6-461e-82f8-285d5d36beb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:45.239839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7941bfd2-d2c6-461e-82f8-285d5d36beb2 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:45.249397 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1401/2832] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:45.253524 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96df41-c30f-4966-a84e-a42bf0c50db3 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:45.265066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96df41-c30f-4966-a84e-a42bf0c50db3 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:45.270338 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce3fad-67ed-4939-9369-04e4271a145d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:45.271018 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce3fad-67ed-4939-9369-04e4271a145d admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:45.271294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce3fad-67ed-4939-9369-04e4271a145d admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:45.271561 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce3fad-67ed-4939-9369-04e4271a145d admin admin] PATH_INFO: `/v3/projects/624d542e011149a7b428605c4405d833/users/be43d129d35b4be281e849ce89912a61/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:45.273480 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce3fad-67ed-4939-9369-04e4271a145d admin admin] RBAC: Authorizing `identity:create_grant(project_id=624d542e011149a7b428605c4405d833, user_id=be43d129d35b4be281e849ce89912a61, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:45.276229 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce3fad-67ed-4939-9369-04e4271a145d admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:45.287309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1432/2833] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:20:45 2022] PUT /identity/v3/projects/624d542e011149a7b428605c4405d833/users/be43d129d35b4be281e849ce89912a61/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:45.298382 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32139f71-5303-4961-9e60-05d2e064cd6e None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:45.298675 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32139f71-5303-4961-9e60-05d2e064cd6e None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:45.298980 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32139f71-5303-4961-9e60-05d2e064cd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:45.328297 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-32139f71-5303-4961-9e60-05d2e064cd6e None None] MFA Rules not processed for user `be43d129d35b4be281e849ce89912a61`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:45.329517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-32139f71-5303-4961-9e60-05d2e064cd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:45.396717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1402/2834] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:45.407493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce3fad-67ed-4939-9369-04e4271a145d admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:45.409144 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d0ce3fad-67ed-4939-9369-04e4271a145d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:45.466057 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce3fad-67ed-4939-9369-04e4271a145d admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:45.469870 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b3714-cfe2-47e4-aa04-4c99367519d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:45.470484 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b3714-cfe2-47e4-aa04-4c99367519d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:45.470769 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b3714-cfe2-47e4-aa04-4c99367519d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:45.471188 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b3714-cfe2-47e4-aa04-4c99367519d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:45.496046 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:45.527918 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b3714-cfe2-47e4-aa04-4c99367519d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:45.529347 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b3714-cfe2-47e4-aa04-4c99367519d8 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:45.538773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1433/2835] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:20:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:20:50.498818 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32139f71-5303-4961-9e60-05d2e064cd6e None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:50.508946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32139f71-5303-4961-9e60-05d2e064cd6e None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:50.512723 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e45da-d0c2-4905-8ecf-68d9ec762de6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:50.513320 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e45da-d0c2-4905-8ecf-68d9ec762de6 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:50.513602 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e45da-d0c2-4905-8ecf-68d9ec762de6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:50.513877 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e45da-d0c2-4905-8ecf-68d9ec762de6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:50.522599 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e45da-d0c2-4905-8ecf-68d9ec762de6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:50.524083 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e45da-d0c2-4905-8ecf-68d9ec762de6 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:50.528949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:50.533560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1403/2836] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:20:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:20:55.555782 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=16 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:56.186171 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b3714-cfe2-47e4-aa04-4c99367519d8 service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:56.246743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b3714-cfe2-47e4-aa04-4c99367519d8 service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:56.251268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5c051-f8a3-41e1-8337-053d66cf871b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:56.251884 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5c051-f8a3-41e1-8337-053d66cf871b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:56.252161 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5c051-f8a3-41e1-8337-053d66cf871b admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:56.252434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5c051-f8a3-41e1-8337-053d66cf871b admin admin] PATH_INFO: `/v3/users/978b3d73cdde49b08913405cc3065c3d` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:56.253540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede5c051-f8a3-41e1-8337-053d66cf871b admin admin] RBAC: Authorizing `identity:delete_user(user_id=978b3d73cdde49b08913405cc3065c3d)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:56.255493 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede5c051-f8a3-41e1-8337-053d66cf871b admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:56.420746 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ede5c051-f8a3-41e1-8337-053d66cf871b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '978b3d73cdde49b08913405cc3065c3d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:56.475594 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ede5c051-f8a3-41e1-8337-053d66cf871b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '978b3d73cdde49b08913405cc3065c3d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:56.488086 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ede5c051-f8a3-41e1-8337-053d66cf871b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '978b3d73cdde49b08913405cc3065c3d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:56.488710 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ede5c051-f8a3-41e1-8337-053d66cf871b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '978b3d73cdde49b08913405cc3065c3d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:56.496105 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ede5c051-f8a3-41e1-8337-053d66cf871b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '978b3d73cdde49b08913405cc3065c3d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:56.501908 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1434/2837] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:20:56 2022] DELETE /identity/v3/users/978b3d73cdde49b08913405cc3065c3d => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:56.506509 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f6ec4-38a4-49d5-b20c-f18bd5ea96aa None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:56.507026 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f6ec4-38a4-49d5-b20c-f18bd5ea96aa None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:56.507215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f6ec4-38a4-49d5-b20c-f18bd5ea96aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:56.525970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-650f6ec4-38a4-49d5-b20c-f18bd5ea96aa None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:56.527202 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-650f6ec4-38a4-49d5-b20c-f18bd5ea96aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:56.584791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1404/2838] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:56.594414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5c051-f8a3-41e1-8337-053d66cf871b admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:56.595847 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ede5c051-f8a3-41e1-8337-053d66cf871b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:56.653990 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5c051-f8a3-41e1-8337-053d66cf871b admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:56.657714 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3747450-ad30-42f1-90de-392c2cbe76cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:56.658324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3747450-ad30-42f1-90de-392c2cbe76cf service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:56.658604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3747450-ad30-42f1-90de-392c2cbe76cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:56.658907 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3747450-ad30-42f1-90de-392c2cbe76cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:56.668529 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3747450-ad30-42f1-90de-392c2cbe76cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:56.669959 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3747450-ad30-42f1-90de-392c2cbe76cf service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:56.679491 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1435/2839] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:20:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:20:57.089268 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f6ec4-38a4-49d5-b20c-f18bd5ea96aa None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:57.090709 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-650f6ec4-38a4-49d5-b20c-f18bd5ea96aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:57.102740 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f6ec4-38a4-49d5-b20c-f18bd5ea96aa None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:57.108079 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767325c8-d852-4b62-ae27-3b4f124831b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:57.108721 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767325c8-d852-4b62-ae27-3b4f124831b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:57.108999 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767325c8-d852-4b62-ae27-3b4f124831b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:57.109270 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767325c8-d852-4b62-ae27-3b4f124831b8 admin admin] PATH_INFO: `/v3/projects/1ebfb382b98d42f18e57a6c310146672` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:57.110399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767325c8-d852-4b62-ae27-3b4f124831b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ebfb382b98d42f18e57a6c310146672)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:57.112366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767325c8-d852-4b62-ae27-3b4f124831b8 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:57.252554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-767325c8-d852-4b62-ae27-3b4f124831b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ebfb382b98d42f18e57a6c310146672', 'request_id': 'req-767325c8-d852-4b62-ae27-3b4f124831b8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:57.257993 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-767325c8-d852-4b62-ae27-3b4f124831b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ebfb382b98d42f18e57a6c310146672', 'request_id': 'req-767325c8-d852-4b62-ae27-3b4f124831b8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:57.343719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-767325c8-d852-4b62-ae27-3b4f124831b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ebfb382b98d42f18e57a6c310146672', 'request_id': 'req-767325c8-d852-4b62-ae27-3b4f124831b8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:57.354996 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-767325c8-d852-4b62-ae27-3b4f124831b8 admin admin] The token cache is being invalidate because project 1ebfb382b98d42f18e57a6c310146672 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:20:57.355248 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-767325c8-d852-4b62-ae27-3b4f124831b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:57.355850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-767325c8-d852-4b62-ae27-3b4f124831b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:57.357338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1405/2840] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:20:57 2022] DELETE /identity/v3/projects/1ebfb382b98d42f18e57a6c310146672 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:57.361423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3747450-ad30-42f1-90de-392c2cbe76cf service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:57.371156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3747450-ad30-42f1-90de-392c2cbe76cf service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:57.389617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:57.427319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac43f6-3501-40b9-a39a-c81a76fe2081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:57.427945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac43f6-3501-40b9-a39a-c81a76fe2081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:57.428227 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac43f6-3501-40b9-a39a-c81a76fe2081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:57.428498 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac43f6-3501-40b9-a39a-c81a76fe2081 admin admin] PATH_INFO: `/v3/users/be43d129d35b4be281e849ce89912a61` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:57.429613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac43f6-3501-40b9-a39a-c81a76fe2081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be43d129d35b4be281e849ce89912a61)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:57.431640 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac43f6-3501-40b9-a39a-c81a76fe2081 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:57.602300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ceac43f6-3501-40b9-a39a-c81a76fe2081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be43d129d35b4be281e849ce89912a61'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:57.658198 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ceac43f6-3501-40b9-a39a-c81a76fe2081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be43d129d35b4be281e849ce89912a61'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:57.671251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ceac43f6-3501-40b9-a39a-c81a76fe2081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be43d129d35b4be281e849ce89912a61'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:57.671911 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ceac43f6-3501-40b9-a39a-c81a76fe2081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be43d129d35b4be281e849ce89912a61'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:57.681894 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-ceac43f6-3501-40b9-a39a-c81a76fe2081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be43d129d35b4be281e849ce89912a61'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:57.692991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1436/2841] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:20:57 2022] DELETE /identity/v3/users/be43d129d35b4be281e849ce89912a61 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:58.055754 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767325c8-d852-4b62-ae27-3b4f124831b8 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:58.057992 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-767325c8-d852-4b62-ae27-3b4f124831b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:58.071134 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767325c8-d852-4b62-ae27-3b4f124831b8 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:58.138690 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760e0b0-05ea-4fe8-9261-f4766f52b70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:58.139287 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760e0b0-05ea-4fe8-9261-f4766f52b70e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:58.139569 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760e0b0-05ea-4fe8-9261-f4766f52b70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:58.139839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760e0b0-05ea-4fe8-9261-f4766f52b70e admin admin] PATH_INFO: `/v3/projects/624d542e011149a7b428605c4405d833` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:58.140949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760e0b0-05ea-4fe8-9261-f4766f52b70e admin admin] RBAC: Authorizing `identity:delete_project(project_id=624d542e011149a7b428605c4405d833)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:58.142970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760e0b0-05ea-4fe8-9261-f4766f52b70e admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:58.259647 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:58.276257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e760e0b0-05ea-4fe8-9261-f4766f52b70e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '624d542e011149a7b428605c4405d833', 'request_id': 'req-e760e0b0-05ea-4fe8-9261-f4766f52b70e'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:58.281598 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e760e0b0-05ea-4fe8-9261-f4766f52b70e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '624d542e011149a7b428605c4405d833', 'request_id': 'req-e760e0b0-05ea-4fe8-9261-f4766f52b70e'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:58.355006 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e760e0b0-05ea-4fe8-9261-f4766f52b70e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '624d542e011149a7b428605c4405d833', 'request_id': 'req-e760e0b0-05ea-4fe8-9261-f4766f52b70e'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:58.366348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e760e0b0-05ea-4fe8-9261-f4766f52b70e admin admin] The token cache is being invalidate because project 624d542e011149a7b428605c4405d833 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:20:58.370412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e760e0b0-05ea-4fe8-9261-f4766f52b70e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:58.370412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-e760e0b0-05ea-4fe8-9261-f4766f52b70e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:20:58.370412 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1406/2842] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:20:58 2022] DELETE /identity/v3/projects/624d542e011149a7b428605c4405d833 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:58.375365 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbf5a3-1f53-4861-9c61-17963e988781 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:58.375661 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbf5a3-1f53-4861-9c61-17963e988781 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:58.375945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbf5a3-1f53-4861-9c61-17963e988781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:58.399083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-b5cbf5a3-1f53-4861-9c61-17963e988781 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:58.399083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-b5cbf5a3-1f53-4861-9c61-17963e988781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:58.455296 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1437/2843] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:58.464155 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632708b-97f1-4295-9b9f-81f76c7e11ed None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:58.464441 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632708b-97f1-4295-9b9f-81f76c7e11ed None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:58.464713 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632708b-97f1-4295-9b9f-81f76c7e11ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:58.484887 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-0632708b-97f1-4295-9b9f-81f76c7e11ed None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:58.486124 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-0632708b-97f1-4295-9b9f-81f76c7e11ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:58.494157 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1407/2844] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:58.498463 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbf5a3-1f53-4861-9c61-17963e988781 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:58.508104 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbf5a3-1f53-4861-9c61-17963e988781 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:58.512920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a984d13-1866-4081-9d44-cf153f5f1e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:58.513507 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a984d13-1866-4081-9d44-cf153f5f1e10 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:58.514967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a984d13-1866-4081-9d44-cf153f5f1e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:58.514967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a984d13-1866-4081-9d44-cf153f5f1e10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:58.514967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a984d13-1866-4081-9d44-cf153f5f1e10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:20:58.514967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a984d13-1866-4081-9d44-cf153f5f1e10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:58.516601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a984d13-1866-4081-9d44-cf153f5f1e10 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:58.528539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1438/2845] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 15:20:58.532777 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632708b-97f1-4295-9b9f-81f76c7e11ed None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:58.542800 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632708b-97f1-4295-9b9f-81f76c7e11ed None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:58.547535 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f9e45-c316-4176-808d-2cd1610fa8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:58.548132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f9e45-c316-4176-808d-2cd1610fa8d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:58.548419 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f9e45-c316-4176-808d-2cd1610fa8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:58.548697 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f9e45-c316-4176-808d-2cd1610fa8d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:58.549360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f9e45-c316-4176-808d-2cd1610fa8d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:58.551282 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f9e45-c316-4176-808d-2cd1610fa8d5 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:58.634506 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1408/2846] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:20:58 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:20:58.635982 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a984d13-1866-4081-9d44-cf153f5f1e10 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:58.646172 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a984d13-1866-4081-9d44-cf153f5f1e10 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:58.672448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:20:58.702177 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fc6d6-15d1-4a05-928e-d016a8998d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:58.702811 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fc6d6-15d1-4a05-928e-d016a8998d3c admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:58.703132 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fc6d6-15d1-4a05-928e-d016a8998d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:58.703408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fc6d6-15d1-4a05-928e-d016a8998d3c admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:58.704041 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fc6d6-15d1-4a05-928e-d016a8998d3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:58.705944 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fc6d6-15d1-4a05-928e-d016a8998d3c admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:58.799232 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1439/2847] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:20:58 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:20:58.800542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f9e45-c316-4176-808d-2cd1610fa8d5 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:58.811717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f9e45-c316-4176-808d-2cd1610fa8d5 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:58.816642 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a661c-c4d3-4508-928c-f073e659f605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:58.821358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a661c-c4d3-4508-928c-f073e659f605 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:58.821358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a661c-c4d3-4508-928c-f073e659f605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:58.821358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a661c-c4d3-4508-928c-f073e659f605 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:58.821358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a661c-c4d3-4508-928c-f073e659f605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:58.821358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a661c-c4d3-4508-928c-f073e659f605 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:58.832010 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1409/2848] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:58.836503 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fc6d6-15d1-4a05-928e-d016a8998d3c admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:58.847738 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fc6d6-15d1-4a05-928e-d016a8998d3c admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:58.852804 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16895d-cadc-4992-991b-7356d078fa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:58.853428 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16895d-cadc-4992-991b-7356d078fa6e admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:58.853740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16895d-cadc-4992-991b-7356d078fa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:58.854010 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16895d-cadc-4992-991b-7356d078fa6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:58.854687 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16895d-cadc-4992-991b-7356d078fa6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:58.856329 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16895d-cadc-4992-991b-7356d078fa6e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:58.871064 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1440/2849] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:20:58.872614 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a661c-c4d3-4508-928c-f073e659f605 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:58.882749 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a661c-c4d3-4508-928c-f073e659f605 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:58.888244 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19682c34-406e-46d5-ae8f-3840693e8381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:58.888899 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19682c34-406e-46d5-ae8f-3840693e8381 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:58.889182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19682c34-406e-46d5-ae8f-3840693e8381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:58.889462 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19682c34-406e-46d5-ae8f-3840693e8381 admin admin] PATH_INFO: `/v3/projects/1dd069b120914a1bb01bc989bcae4405/users/6dca169db15c4cd4bc643f72adb32c3a/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:58.904454 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19682c34-406e-46d5-ae8f-3840693e8381 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dd069b120914a1bb01bc989bcae4405, user_id=6dca169db15c4cd4bc643f72adb32c3a, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:58.907119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19682c34-406e-46d5-ae8f-3840693e8381 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:58.971946 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1410/2850] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:20:58 2022] PUT /identity/v3/projects/1dd069b120914a1bb01bc989bcae4405/users/6dca169db15c4cd4bc643f72adb32c3a/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:20:58.983311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299249bb-4499-445f-bdbb-d4d15251e0b1 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:58.983597 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299249bb-4499-445f-bdbb-d4d15251e0b1 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:58.983899 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299249bb-4499-445f-bdbb-d4d15251e0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:59.012075 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-299249bb-4499-445f-bdbb-d4d15251e0b1 None None] MFA Rules not processed for user `6dca169db15c4cd4bc643f72adb32c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:20:59.013284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-299249bb-4499-445f-bdbb-d4d15251e0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:59.078626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1441/2851] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:20:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:20:59.089457 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19682c34-406e-46d5-ae8f-3840693e8381 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:20:59.091194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-19682c34-406e-46d5-ae8f-3840693e8381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:20:59.148366 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19682c34-406e-46d5-ae8f-3840693e8381 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:20:59.152080 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f706f-6731-45ee-946d-e79b748ae1df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:20:59.156863 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f706f-6731-45ee-946d-e79b748ae1df service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:20:59.156863 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f706f-6731-45ee-946d-e79b748ae1df service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:20:59.156863 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f706f-6731-45ee-946d-e79b748ae1df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:20:59.161451 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f706f-6731-45ee-946d-e79b748ae1df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:20:59.162959 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f706f-6731-45ee-946d-e79b748ae1df service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:20:59.171749 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1411/2852] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:20:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:21:09.075744 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:21:09.169511 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:21:10.222086 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299249bb-4499-445f-bdbb-d4d15251e0b1 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:21:10.283825 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299249bb-4499-445f-bdbb-d4d15251e0b1 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:21:10.288223 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347196c3-c67c-4371-898d-f669abf4bff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:21:10.288841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347196c3-c67c-4371-898d-f669abf4bff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:21:10.289120 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347196c3-c67c-4371-898d-f669abf4bff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:21:10.289391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347196c3-c67c-4371-898d-f669abf4bff5 admin admin] PATH_INFO: `/v3/users/6dca169db15c4cd4bc643f72adb32c3a` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:21:10.291108 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347196c3-c67c-4371-898d-f669abf4bff5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dca169db15c4cd4bc643f72adb32c3a)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:21:10.293006 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347196c3-c67c-4371-898d-f669abf4bff5 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:21:10.436647 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-347196c3-c67c-4371-898d-f669abf4bff5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dca169db15c4cd4bc643f72adb32c3a'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:21:10.509582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-347196c3-c67c-4371-898d-f669abf4bff5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dca169db15c4cd4bc643f72adb32c3a'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:21:10.522383 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-347196c3-c67c-4371-898d-f669abf4bff5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dca169db15c4cd4bc643f72adb32c3a'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:21:10.525143 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-347196c3-c67c-4371-898d-f669abf4bff5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dca169db15c4cd4bc643f72adb32c3a'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:21:10.530648 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-347196c3-c67c-4371-898d-f669abf4bff5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dca169db15c4cd4bc643f72adb32c3a'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:21:10.536807 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1442/2853] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:21:10 2022] DELETE /identity/v3/users/6dca169db15c4cd4bc643f72adb32c3a => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:21:10.541773 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378f28c-7ddd-4cb7-994d-e985166cfdf2 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:21:10.542228 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378f28c-7ddd-4cb7-994d-e985166cfdf2 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:21:10.542333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378f28c-7ddd-4cb7-994d-e985166cfdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:21:10.566125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-7378f28c-7ddd-4cb7-994d-e985166cfdf2 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:21:10.567357 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-7378f28c-7ddd-4cb7-994d-e985166cfdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:21:10.625430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1412/2854] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 15:21:10.635614 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347196c3-c67c-4371-898d-f669abf4bff5 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:21:10.637400 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-347196c3-c67c-4371-898d-f669abf4bff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:21:10.693957 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347196c3-c67c-4371-898d-f669abf4bff5 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:21:10.698028 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a482c-b5e1-4137-909a-ba7bbaf9573b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:21:10.698631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a482c-b5e1-4137-909a-ba7bbaf9573b service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:21:10.698946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a482c-b5e1-4137-909a-ba7bbaf9573b service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:21:10.699224 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a482c-b5e1-4137-909a-ba7bbaf9573b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:21:10.708047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a482c-b5e1-4137-909a-ba7bbaf9573b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:21:10.709476 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a482c-b5e1-4137-909a-ba7bbaf9573b service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:21:10.719461 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1443/2855] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 15:21:11.094000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378f28c-7ddd-4cb7-994d-e985166cfdf2 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:21:11.095461 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-7378f28c-7ddd-4cb7-994d-e985166cfdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:21:11.109794 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378f28c-7ddd-4cb7-994d-e985166cfdf2 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:21:11.115054 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d47e5-1721-4879-bd7b-477f52206582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:21:11.115271 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d47e5-1721-4879-bd7b-477f52206582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:21:11.115524 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d47e5-1721-4879-bd7b-477f52206582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:21:11.115800 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d47e5-1721-4879-bd7b-477f52206582 admin admin] PATH_INFO: `/v3/projects/1dd069b120914a1bb01bc989bcae4405` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:21:11.116940 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d47e5-1721-4879-bd7b-477f52206582 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dd069b120914a1bb01bc989bcae4405)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:21:11.119389 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d47e5-1721-4879-bd7b-477f52206582 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:21:11.195469 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2e3d47e5-1721-4879-bd7b-477f52206582 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dd069b120914a1bb01bc989bcae4405', 'request_id': 'req-2e3d47e5-1721-4879-bd7b-477f52206582'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:21:11.201603 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2e3d47e5-1721-4879-bd7b-477f52206582 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dd069b120914a1bb01bc989bcae4405', 'request_id': 'req-2e3d47e5-1721-4879-bd7b-477f52206582'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:21:11.303926 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2e3d47e5-1721-4879-bd7b-477f52206582 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dd069b120914a1bb01bc989bcae4405', 'request_id': 'req-2e3d47e5-1721-4879-bd7b-477f52206582'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:21:11.315928 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2e3d47e5-1721-4879-bd7b-477f52206582 admin admin] The token cache is being invalidate because project 1dd069b120914a1bb01bc989bcae4405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:21:11.316215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2e3d47e5-1721-4879-bd7b-477f52206582 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:21:11.316828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-2e3d47e5-1721-4879-bd7b-477f52206582 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:21:11.318293 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1413/2856] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:21:11 2022] DELETE /identity/v3/projects/1dd069b120914a1bb01bc989bcae4405 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:21:20.715309 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:21:21.312935 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:02.645209 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a482c-b5e1-4137-909a-ba7bbaf9573b service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:02.646604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d34a482c-b5e1-4137-909a-ba7bbaf9573b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:02.702473 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a482c-b5e1-4137-909a-ba7bbaf9573b service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:02.749057 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d1be7-6540-49a1-96a6-0681080cee9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:02.749666 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d1be7-6540-49a1-96a6-0681080cee9a service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:02.749946 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d1be7-6540-49a1-96a6-0681080cee9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:02.750217 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d1be7-6540-49a1-96a6-0681080cee9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:02.752530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-928d1be7-6540-49a1-96a6-0681080cee9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:02.812154 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d1be7-6540-49a1-96a6-0681080cee9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:02.813598 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d1be7-6540-49a1-96a6-0681080cee9a service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:02.832731 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1444/2857] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:03.007753 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d47e5-1721-4879-bd7b-477f52206582 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:03.009137 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-2e3d47e5-1721-4879-bd7b-477f52206582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:03.066092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d47e5-1721-4879-bd7b-477f52206582 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:03.069751 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eab5ab-f92d-4acb-bc80-5393238462b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:03.070394 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eab5ab-f92d-4acb-bc80-5393238462b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:03.070679 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eab5ab-f92d-4acb-bc80-5393238462b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:03.071096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eab5ab-f92d-4acb-bc80-5393238462b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:03.072821 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-19eab5ab-f92d-4acb-bc80-5393238462b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:03.134853 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eab5ab-f92d-4acb-bc80-5393238462b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:03.136347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eab5ab-f92d-4acb-bc80-5393238462b9 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:03.146546 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1414/2858] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:06.643029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d1be7-6540-49a1-96a6-0681080cee9a service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:06.644477 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-928d1be7-6540-49a1-96a6-0681080cee9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:06.711740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=100 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:06.716137 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d1be7-6540-49a1-96a6-0681080cee9a service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:06.721739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ee556-1036-4c82-a8cd-f4b975a2c860 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:06.722345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ee556-1036-4c82-a8cd-f4b975a2c860 service placement] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:06.722627 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ee556-1036-4c82-a8cd-f4b975a2c860 service placement] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:06.722933 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ee556-1036-4c82-a8cd-f4b975a2c860 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:06.726631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-9f2ee556-1036-4c82-a8cd-f4b975a2c860 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:06.736380 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ee556-1036-4c82-a8cd-f4b975a2c860 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:06.737960 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ee556-1036-4c82-a8cd-f4b975a2c860 service placement] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:06.748052 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1445/2859] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:06.761810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eab5ab-f92d-4acb-bc80-5393238462b9 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:06.763275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-19eab5ab-f92d-4acb-bc80-5393238462b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:06.811696 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eab5ab-f92d-4acb-bc80-5393238462b9 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:06.816329 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e75261-8696-4774-85cc-86981c046a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:06.816976 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e75261-8696-4774-85cc-86981c046a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:06.817255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e75261-8696-4774-85cc-86981c046a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:06.817550 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e75261-8696-4774-85cc-86981c046a9f admin admin] PATH_INFO: `/v3/users/91c691b6a7724d708bfc7424908516c8` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:06.818674 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e75261-8696-4774-85cc-86981c046a9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=91c691b6a7724d708bfc7424908516c8)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:06.820636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e75261-8696-4774-85cc-86981c046a9f admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:06.993991 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-c1e75261-8696-4774-85cc-86981c046a9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91c691b6a7724d708bfc7424908516c8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:07.055151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-c1e75261-8696-4774-85cc-86981c046a9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91c691b6a7724d708bfc7424908516c8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:07.055756 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-c1e75261-8696-4774-85cc-86981c046a9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91c691b6a7724d708bfc7424908516c8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:07.068223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-c1e75261-8696-4774-85cc-86981c046a9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91c691b6a7724d708bfc7424908516c8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:07.072650 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:07.075680 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-c1e75261-8696-4774-85cc-86981c046a9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91c691b6a7724d708bfc7424908516c8'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:07.082280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1415/2860] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:23:06 2022] DELETE /identity/v3/users/91c691b6a7724d708bfc7424908516c8 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:23:07.356326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ee556-1036-4c82-a8cd-f4b975a2c860 service placement] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:07.367448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ee556-1036-4c82-a8cd-f4b975a2c860 service placement] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:07.422289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83294d19-3135-4991-84cd-cc78520efcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:07.423430 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83294d19-3135-4991-84cd-cc78520efcb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:07.423718 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83294d19-3135-4991-84cd-cc78520efcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:07.423987 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83294d19-3135-4991-84cd-cc78520efcb2 admin admin] PATH_INFO: `/v3/projects/834730ab3cfd4d649fc270843d47b291` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:07.425083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83294d19-3135-4991-84cd-cc78520efcb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=834730ab3cfd4d649fc270843d47b291)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:07.426994 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83294d19-3135-4991-84cd-cc78520efcb2 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:07.524382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-83294d19-3135-4991-84cd-cc78520efcb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '834730ab3cfd4d649fc270843d47b291', 'request_id': 'req-83294d19-3135-4991-84cd-cc78520efcb2'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:07.531156 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-83294d19-3135-4991-84cd-cc78520efcb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '834730ab3cfd4d649fc270843d47b291', 'request_id': 'req-83294d19-3135-4991-84cd-cc78520efcb2'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:07.609540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-83294d19-3135-4991-84cd-cc78520efcb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '834730ab3cfd4d649fc270843d47b291', 'request_id': 'req-83294d19-3135-4991-84cd-cc78520efcb2'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:07.621068 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-83294d19-3135-4991-84cd-cc78520efcb2 admin admin] The token cache is being invalidate because project 834730ab3cfd4d649fc270843d47b291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:23:07.624574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-83294d19-3135-4991-84cd-cc78520efcb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:07.624574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-83294d19-3135-4991-84cd-cc78520efcb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:07.624574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1446/2861] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:23:07 2022] DELETE /identity/v3/projects/834730ab3cfd4d649fc270843d47b291 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:07.629899 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bb9b4-7c17-4f56-9bd3-c31d58a9627c None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:07.630183 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bb9b4-7c17-4f56-9bd3-c31d58a9627c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:07.630476 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bb9b4-7c17-4f56-9bd3-c31d58a9627c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:07.689781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-435bb9b4-7c17-4f56-9bd3-c31d58a9627c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:07.691016 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-435bb9b4-7c17-4f56-9bd3-c31d58a9627c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:07.747637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1416/2862] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:07.755238 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43a1e8-07bb-4766-92a4-a9b3578910c1 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:07.755694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43a1e8-07bb-4766-92a4-a9b3578910c1 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:07.755808 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43a1e8-07bb-4766-92a4-a9b3578910c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:07.775515 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-bb43a1e8-07bb-4766-92a4-a9b3578910c1 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:07.776726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-bb43a1e8-07bb-4766-92a4-a9b3578910c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:07.785043 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1447/2863] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:07.789105 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435bb9b4-7c17-4f56-9bd3-c31d58a9627c None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:07.799477 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435bb9b4-7c17-4f56-9bd3-c31d58a9627c None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:07.804089 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09d077-f803-42a9-ac76-759e5c4a8db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:07.804678 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09d077-f803-42a9-ac76-759e5c4a8db5 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:07.805031 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09d077-f803-42a9-ac76-759e5c4a8db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:07.805329 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09d077-f803-42a9-ac76-759e5c4a8db5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:07.806296 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09d077-f803-42a9-ac76-759e5c4a8db5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:23:07.806608 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09d077-f803-42a9-ac76-759e5c4a8db5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:07.808242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09d077-f803-42a9-ac76-759e5c4a8db5 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:07.819533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1417/2864] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 15:23:07.823754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43a1e8-07bb-4766-92a4-a9b3578910c1 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:07.833011 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43a1e8-07bb-4766-92a4-a9b3578910c1 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:07.837677 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c539f-9442-451f-b37f-fd7b5f6e622d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:07.838271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c539f-9442-451f-b37f-fd7b5f6e622d admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:07.838555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c539f-9442-451f-b37f-fd7b5f6e622d admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:07.838866 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c539f-9442-451f-b37f-fd7b5f6e622d admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:07.839508 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c539f-9442-451f-b37f-fd7b5f6e622d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:07.841408 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c539f-9442-451f-b37f-fd7b5f6e622d admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:07.922157 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1448/2865] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:23:07 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:23:07.927213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09d077-f803-42a9-ac76-759e5c4a8db5 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:07.937329 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09d077-f803-42a9-ac76-759e5c4a8db5 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:07.992347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865788d5-315e-4ad7-8385-62324ef3be89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:07.992954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865788d5-315e-4ad7-8385-62324ef3be89 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:07.993237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865788d5-315e-4ad7-8385-62324ef3be89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:07.993516 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865788d5-315e-4ad7-8385-62324ef3be89 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:07.994139 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865788d5-315e-4ad7-8385-62324ef3be89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:07.996096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865788d5-315e-4ad7-8385-62324ef3be89 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:08.062705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1418/2866] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:23:07 2022] POST /identity/v3/users => generated 288 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:23:08.072054 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c539f-9442-451f-b37f-fd7b5f6e622d admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:08.082853 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c539f-9442-451f-b37f-fd7b5f6e622d admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:08.087767 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f640bd-8b62-40d6-b716-7cbeb7bebcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:08.088402 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f640bd-8b62-40d6-b716-7cbeb7bebcb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:08.088688 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f640bd-8b62-40d6-b716-7cbeb7bebcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:08.088966 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f640bd-8b62-40d6-b716-7cbeb7bebcb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:08.089606 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f640bd-8b62-40d6-b716-7cbeb7bebcb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:08.091355 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f640bd-8b62-40d6-b716-7cbeb7bebcb0 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:08.102038 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1449/2867] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:23:08.106073 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865788d5-315e-4ad7-8385-62324ef3be89 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:08.116752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865788d5-315e-4ad7-8385-62324ef3be89 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:08.121466 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265af9ba-afb6-4b6e-b571-9b1184c6a2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:08.122095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265af9ba-afb6-4b6e-b571-9b1184c6a2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:08.122382 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265af9ba-afb6-4b6e-b571-9b1184c6a2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:08.122660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265af9ba-afb6-4b6e-b571-9b1184c6a2d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:08.123333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265af9ba-afb6-4b6e-b571-9b1184c6a2d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:08.124933 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265af9ba-afb6-4b6e-b571-9b1184c6a2d6 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:08.135391 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1419/2868] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:23:08.139595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f640bd-8b62-40d6-b716-7cbeb7bebcb0 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:08.149974 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f640bd-8b62-40d6-b716-7cbeb7bebcb0 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:08.154553 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd0c61-2f97-4ff9-90d4-ee157618ec93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:08.155236 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd0c61-2f97-4ff9-90d4-ee157618ec93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:08.155525 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd0c61-2f97-4ff9-90d4-ee157618ec93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:08.155803 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd0c61-2f97-4ff9-90d4-ee157618ec93 admin admin] PATH_INFO: `/v3/projects/eb5447bb1f444ff1a5de329c02db774b/users/c038f043eda14202b4d5594b51027220/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:08.170820 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd0c61-2f97-4ff9-90d4-ee157618ec93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb5447bb1f444ff1a5de329c02db774b, user_id=c038f043eda14202b4d5594b51027220, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:08.173495 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd0c61-2f97-4ff9-90d4-ee157618ec93 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:08.280539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1450/2869] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:23:08 2022] PUT /identity/v3/projects/eb5447bb1f444ff1a5de329c02db774b/users/c038f043eda14202b4d5594b51027220/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:08.292724 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84543afc-91b1-46f4-9ceb-05f43d459f21 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:08.293015 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84543afc-91b1-46f4-9ceb-05f43d459f21 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:08.293294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84543afc-91b1-46f4-9ceb-05f43d459f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:08.320481 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-84543afc-91b1-46f4-9ceb-05f43d459f21 None None] MFA Rules not processed for user `c038f043eda14202b4d5594b51027220`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:08.321689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-84543afc-91b1-46f4-9ceb-05f43d459f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:08.338750 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:08.386059 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1420/2870] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:08.405423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd0c61-2f97-4ff9-90d4-ee157618ec93 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:08.405423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-74bd0c61-2f97-4ff9-90d4-ee157618ec93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:08.414544 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:08.476391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd0c61-2f97-4ff9-90d4-ee157618ec93 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:08.482017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa70eb-09c3-422d-a108-ba9fbcbfee00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:08.482620 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa70eb-09c3-422d-a108-ba9fbcbfee00 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:08.482939 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa70eb-09c3-422d-a108-ba9fbcbfee00 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:08.483215 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa70eb-09c3-422d-a108-ba9fbcbfee00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:08.490709 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa70eb-09c3-422d-a108-ba9fbcbfee00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:08.492304 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa70eb-09c3-422d-a108-ba9fbcbfee00 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:08.500903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1451/2871] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:08.583692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84543afc-91b1-46f4-9ceb-05f43d459f21 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:08.585101 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-84543afc-91b1-46f4-9ceb-05f43d459f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:08.640503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84543afc-91b1-46f4-9ceb-05f43d459f21 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:08.644939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d06279-be7a-45c7-b717-d3d701e3c4b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:08.645543 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d06279-be7a-45c7-b717-d3d701e3c4b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:08.645825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d06279-be7a-45c7-b717-d3d701e3c4b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:08.646098 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d06279-be7a-45c7-b717-d3d701e3c4b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:08.647951 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-52d06279-be7a-45c7-b717-d3d701e3c4b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:08.661552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d06279-be7a-45c7-b717-d3d701e3c4b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:08.662967 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d06279-be7a-45c7-b717-d3d701e3c4b9 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:08.673964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1421/2872] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:08.744856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa70eb-09c3-422d-a108-ba9fbcbfee00 service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:08.746330 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-05fa70eb-09c3-422d-a108-ba9fbcbfee00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:08.758620 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa70eb-09c3-422d-a108-ba9fbcbfee00 service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:08.762137 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2c669-0a73-44b1-9b4d-8aeaac32845d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:08.762723 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2c669-0a73-44b1-9b4d-8aeaac32845d service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:08.763029 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2c669-0a73-44b1-9b4d-8aeaac32845d service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:08.763300 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2c669-0a73-44b1-9b4d-8aeaac32845d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:08.765171 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-34d2c669-0a73-44b1-9b4d-8aeaac32845d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:08.778707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2c669-0a73-44b1-9b4d-8aeaac32845d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:08.780155 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2c669-0a73-44b1-9b4d-8aeaac32845d service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:08.791637 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1452/2873] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:08.867340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d06279-be7a-45c7-b717-d3d701e3c4b9 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:08.928987 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d06279-be7a-45c7-b717-d3d701e3c4b9 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:08.933399 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ab641-23ea-4501-91cd-607c8d6f60dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:08.934049 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ab641-23ea-4501-91cd-607c8d6f60dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:08.934333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ab641-23ea-4501-91cd-607c8d6f60dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:08.934605 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ab641-23ea-4501-91cd-607c8d6f60dc admin admin] PATH_INFO: `/v3/users/c038f043eda14202b4d5594b51027220` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:08.935765 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ab641-23ea-4501-91cd-607c8d6f60dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c038f043eda14202b4d5594b51027220)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:08.937657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ab641-23ea-4501-91cd-607c8d6f60dc admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:09.103599 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8d1ab641-23ea-4501-91cd-607c8d6f60dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c038f043eda14202b4d5594b51027220'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:09.163550 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8d1ab641-23ea-4501-91cd-607c8d6f60dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c038f043eda14202b4d5594b51027220'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:09.164166 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8d1ab641-23ea-4501-91cd-607c8d6f60dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c038f043eda14202b4d5594b51027220'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:09.171485 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:09.177978 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8d1ab641-23ea-4501-91cd-607c8d6f60dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c038f043eda14202b4d5594b51027220'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:09.184823 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-8d1ab641-23ea-4501-91cd-607c8d6f60dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c038f043eda14202b4d5594b51027220'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:09.191348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1422/2874] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:23:08 2022] DELETE /identity/v3/users/c038f043eda14202b4d5594b51027220 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:09.195922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d72cb1-48a8-4eac-acac-ef66343652ca None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:09.199251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d72cb1-48a8-4eac-acac-ef66343652ca None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:09.199251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d72cb1-48a8-4eac-acac-ef66343652ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:09.216021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-85d72cb1-48a8-4eac-acac-ef66343652ca None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:09.217210 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-85d72cb1-48a8-4eac-acac-ef66343652ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:09.275945 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1453/2875] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:09.286283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ab641-23ea-4501-91cd-607c8d6f60dc admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:09.287712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8d1ab641-23ea-4501-91cd-607c8d6f60dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:09.356845 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ab641-23ea-4501-91cd-607c8d6f60dc admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:09.360997 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b42a7f-7fc5-430a-b685-f34db5495f6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:09.361689 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b42a7f-7fc5-430a-b685-f34db5495f6e service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:09.361970 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b42a7f-7fc5-430a-b685-f34db5495f6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:09.362246 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b42a7f-7fc5-430a-b685-f34db5495f6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:09.371185 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b42a7f-7fc5-430a-b685-f34db5495f6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:09.372658 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b42a7f-7fc5-430a-b685-f34db5495f6e service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:09.383239 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1423/2876] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:09.799555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d72cb1-48a8-4eac-acac-ef66343652ca None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:09.799555 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-85d72cb1-48a8-4eac-acac-ef66343652ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:09.807088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d72cb1-48a8-4eac-acac-ef66343652ca None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:09.811955 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527fb95b-d398-4b65-9ca4-bd55f2ad1924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:09.812604 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527fb95b-d398-4b65-9ca4-bd55f2ad1924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:09.812887 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527fb95b-d398-4b65-9ca4-bd55f2ad1924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:09.813159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527fb95b-d398-4b65-9ca4-bd55f2ad1924 admin admin] PATH_INFO: `/v3/projects/eb5447bb1f444ff1a5de329c02db774b` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:09.814414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527fb95b-d398-4b65-9ca4-bd55f2ad1924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb5447bb1f444ff1a5de329c02db774b)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:09.816585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527fb95b-d398-4b65-9ca4-bd55f2ad1924 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:09.910360 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-527fb95b-d398-4b65-9ca4-bd55f2ad1924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb5447bb1f444ff1a5de329c02db774b', 'request_id': 'req-527fb95b-d398-4b65-9ca4-bd55f2ad1924'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:09.916504 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-527fb95b-d398-4b65-9ca4-bd55f2ad1924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb5447bb1f444ff1a5de329c02db774b', 'request_id': 'req-527fb95b-d398-4b65-9ca4-bd55f2ad1924'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:09.974934 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-527fb95b-d398-4b65-9ca4-bd55f2ad1924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb5447bb1f444ff1a5de329c02db774b', 'request_id': 'req-527fb95b-d398-4b65-9ca4-bd55f2ad1924'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:09.985496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-527fb95b-d398-4b65-9ca4-bd55f2ad1924 admin admin] The token cache is being invalidate because project eb5447bb1f444ff1a5de329c02db774b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:23:09.985782 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-527fb95b-d398-4b65-9ca4-bd55f2ad1924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:09.986404 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-527fb95b-d398-4b65-9ca4-bd55f2ad1924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:09.990068 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1454/2877] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:23:09 2022] DELETE /identity/v3/projects/eb5447bb1f444ff1a5de329c02db774b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:09.992770 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbab6c1-8579-4fce-9d41-d6a366626c49 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:09.993235 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbab6c1-8579-4fce-9d41-d6a366626c49 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:09.993359 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbab6c1-8579-4fce-9d41-d6a366626c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:10.013132 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-7cbab6c1-8579-4fce-9d41-d6a366626c49 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:10.014668 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-7cbab6c1-8579-4fce-9d41-d6a366626c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:10.072112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1424/2878] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:10.080622 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0809d3e-376f-41e7-abb0-ecd3ed165ad0 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:10.080906 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0809d3e-376f-41e7-abb0-ecd3ed165ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:10.081176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0809d3e-376f-41e7-abb0-ecd3ed165ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:10.091101 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:10.102243 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-f0809d3e-376f-41e7-abb0-ecd3ed165ad0 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:10.103482 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-f0809d3e-376f-41e7-abb0-ecd3ed165ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:10.111490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1455/2879] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:10.115695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbab6c1-8579-4fce-9d41-d6a366626c49 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:10.125190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbab6c1-8579-4fce-9d41-d6a366626c49 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:10.130187 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1c447-2c07-483a-98bc-7be44aa9c0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:10.130779 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1c447-2c07-483a-98bc-7be44aa9c0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:10.131091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1c447-2c07-483a-98bc-7be44aa9c0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:10.131362 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1c447-2c07-483a-98bc-7be44aa9c0a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:10.131998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1c447-2c07-483a-98bc-7be44aa9c0a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:23:10.132296 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1c447-2c07-483a-98bc-7be44aa9c0a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:10.133875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1c447-2c07-483a-98bc-7be44aa9c0a8 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:10.145961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1425/2880] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 15:23:10.150091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0809d3e-376f-41e7-abb0-ecd3ed165ad0 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:10.159326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0809d3e-376f-41e7-abb0-ecd3ed165ad0 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:10.164004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfc2da-0e7f-459d-b752-2dd741fa54a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:10.164590 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfc2da-0e7f-459d-b752-2dd741fa54a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:10.164871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfc2da-0e7f-459d-b752-2dd741fa54a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:10.165153 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfc2da-0e7f-459d-b752-2dd741fa54a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:10.165830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dfc2da-0e7f-459d-b752-2dd741fa54a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:10.167756 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dfc2da-0e7f-459d-b752-2dd741fa54a4 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:10.258848 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1456/2881] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:23:10 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:23:10.262876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1c447-2c07-483a-98bc-7be44aa9c0a8 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:10.273599 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1c447-2c07-483a-98bc-7be44aa9c0a8 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:10.327997 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7fd53-46fb-4e6d-ab56-ebeb15f12e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:10.328528 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7fd53-46fb-4e6d-ab56-ebeb15f12e68 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:10.328811 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7fd53-46fb-4e6d-ab56-ebeb15f12e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:10.329085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7fd53-46fb-4e6d-ab56-ebeb15f12e68 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:10.330286 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7fd53-46fb-4e6d-ab56-ebeb15f12e68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:10.332242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7fd53-46fb-4e6d-ab56-ebeb15f12e68 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:10.340146 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:10.412085 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1426/2882] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:23:10 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:23:10.416269 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfc2da-0e7f-459d-b752-2dd741fa54a4 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:10.427070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfc2da-0e7f-459d-b752-2dd741fa54a4 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:10.431638 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484b088-41a2-4873-ae26-e5c8b6fed19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:10.432226 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484b088-41a2-4873-ae26-e5c8b6fed19d admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:10.432502 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484b088-41a2-4873-ae26-e5c8b6fed19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:10.432773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484b088-41a2-4873-ae26-e5c8b6fed19d admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:10.433442 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484b088-41a2-4873-ae26-e5c8b6fed19d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:10.435055 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484b088-41a2-4873-ae26-e5c8b6fed19d admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:10.445448 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1457/2883] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:23:10.449210 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7fd53-46fb-4e6d-ab56-ebeb15f12e68 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:10.458959 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7fd53-46fb-4e6d-ab56-ebeb15f12e68 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:10.463489 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee261f-1712-4757-ac2f-78dd5e8efce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:10.464072 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee261f-1712-4757-ac2f-78dd5e8efce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:10.464353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee261f-1712-4757-ac2f-78dd5e8efce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:10.464626 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee261f-1712-4757-ac2f-78dd5e8efce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:10.465255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee261f-1712-4757-ac2f-78dd5e8efce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:10.466903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee261f-1712-4757-ac2f-78dd5e8efce1 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:10.477225 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1427/2884] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 15:23:10.481097 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484b088-41a2-4873-ae26-e5c8b6fed19d admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:10.491319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484b088-41a2-4873-ae26-e5c8b6fed19d admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:10.495885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6e3f1-20b1-429a-b6c0-da5140eae180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:10.496514 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6e3f1-20b1-429a-b6c0-da5140eae180 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:10.496792 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6e3f1-20b1-429a-b6c0-da5140eae180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:10.497061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6e3f1-20b1-429a-b6c0-da5140eae180 admin admin] PATH_INFO: `/v3/projects/8a7f780589ea42a0ab619506a9e1cbb7/users/9d8f221166664e67b40249b3f53aa974/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:10.511841 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6e3f1-20b1-429a-b6c0-da5140eae180 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7f780589ea42a0ab619506a9e1cbb7, user_id=9d8f221166664e67b40249b3f53aa974, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:10.514922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6e3f1-20b1-429a-b6c0-da5140eae180 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:10.577370 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1458/2885] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:23:10 2022] PUT /identity/v3/projects/8a7f780589ea42a0ab619506a9e1cbb7/users/9d8f221166664e67b40249b3f53aa974/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:23:10.591200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81f82e-341f-4ec1-bc34-2495539319cd None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:10.591200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81f82e-341f-4ec1-bc34-2495539319cd None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:10.591200 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81f82e-341f-4ec1-bc34-2495539319cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:10.614828 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-9e81f82e-341f-4ec1-bc34-2495539319cd None None] MFA Rules not processed for user `9d8f221166664e67b40249b3f53aa974`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:10.616019 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-9e81f82e-341f-4ec1-bc34-2495539319cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:10.677555 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1428/2886] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:10.681887 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6e3f1-20b1-429a-b6c0-da5140eae180 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:10.743312 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6e3f1-20b1-429a-b6c0-da5140eae180 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:10.748133 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608029c-31b6-425b-9d34-fd2976461786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:10.748744 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608029c-31b6-425b-9d34-fd2976461786 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:10.749027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608029c-31b6-425b-9d34-fd2976461786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:10.749308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608029c-31b6-425b-9d34-fd2976461786 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:10.749942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8608029c-31b6-425b-9d34-fd2976461786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:10.751931 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8608029c-31b6-425b-9d34-fd2976461786 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:10.856492 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1459/2887] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:23:10 2022] POST /identity/v3/projects => generated 381 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:23:10.861189 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81f82e-341f-4ec1-bc34-2495539319cd None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:10.871874 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81f82e-341f-4ec1-bc34-2495539319cd None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:10.927435 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e391b-c7ba-4f57-9141-864f720757ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:10.928029 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e391b-c7ba-4f57-9141-864f720757ee admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:10.928313 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e391b-c7ba-4f57-9141-864f720757ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:10.928582 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e391b-c7ba-4f57-9141-864f720757ee admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:10.929193 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e391b-c7ba-4f57-9141-864f720757ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:10.931131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e391b-c7ba-4f57-9141-864f720757ee admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:10.992856 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1429/2888] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:23:10 2022] POST /identity/v3/users => generated 297 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:23:10.997066 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608029c-31b6-425b-9d34-fd2976461786 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:11.007395 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608029c-31b6-425b-9d34-fd2976461786 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:11.012018 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc52523-bb1f-4224-8e8c-a18cabde1152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:11.012625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc52523-bb1f-4224-8e8c-a18cabde1152 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:11.012892 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc52523-bb1f-4224-8e8c-a18cabde1152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:11.013163 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc52523-bb1f-4224-8e8c-a18cabde1152 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:11.014050 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc52523-bb1f-4224-8e8c-a18cabde1152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:11.015737 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc52523-bb1f-4224-8e8c-a18cabde1152 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:11.026483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1460/2889] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:23:11.030373 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e391b-c7ba-4f57-9141-864f720757ee admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:11.040583 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e391b-c7ba-4f57-9141-864f720757ee admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:11.045128 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10fb49-474e-4481-84bf-fe3dcfb0c664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:11.045767 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10fb49-474e-4481-84bf-fe3dcfb0c664 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:11.046040 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10fb49-474e-4481-84bf-fe3dcfb0c664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:11.046309 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10fb49-474e-4481-84bf-fe3dcfb0c664 admin admin] PATH_INFO: `/v3/projects/cc672b0a96954688aca2a47fabb1de3b/users/66510ba7a24246efb73503326ebac404/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:11.061282 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10fb49-474e-4481-84bf-fe3dcfb0c664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc672b0a96954688aca2a47fabb1de3b, user_id=66510ba7a24246efb73503326ebac404, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:11.063957 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10fb49-474e-4481-84bf-fe3dcfb0c664 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:11.147809 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1430/2890] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:23:11 2022] PUT /identity/v3/projects/cc672b0a96954688aca2a47fabb1de3b/users/66510ba7a24246efb73503326ebac404/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:11.151881 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc52523-bb1f-4224-8e8c-a18cabde1152 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:11.162419 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc52523-bb1f-4224-8e8c-a18cabde1152 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:11.216935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0f68d-e91b-4090-bcf7-aade2fc831ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:11.217539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0f68d-e91b-4090-bcf7-aade2fc831ab admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:11.217827 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0f68d-e91b-4090-bcf7-aade2fc831ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:11.218101 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0f68d-e91b-4090-bcf7-aade2fc831ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:11.218735 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0f68d-e91b-4090-bcf7-aade2fc831ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:11.221002 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0f68d-e91b-4090-bcf7-aade2fc831ab admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:11.230471 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1461/2891] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:23:11.234875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10fb49-474e-4481-84bf-fe3dcfb0c664 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:11.245009 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10fb49-474e-4481-84bf-fe3dcfb0c664 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:11.249607 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a61b2-be04-4209-b3d5-4be41da8183f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:11.253525 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a61b2-be04-4209-b3d5-4be41da8183f admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:11.253525 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a61b2-be04-4209-b3d5-4be41da8183f admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:11.253525 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a61b2-be04-4209-b3d5-4be41da8183f admin admin] PATH_INFO: `/v3/projects/cc672b0a96954688aca2a47fabb1de3b/users/66510ba7a24246efb73503326ebac404/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:11.253525 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a61b2-be04-4209-b3d5-4be41da8183f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc672b0a96954688aca2a47fabb1de3b, user_id=66510ba7a24246efb73503326ebac404, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:11.255372 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a61b2-be04-4209-b3d5-4be41da8183f admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:11.265572 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1431/2892] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:23:11 2022] PUT /identity/v3/projects/cc672b0a96954688aca2a47fabb1de3b/users/66510ba7a24246efb73503326ebac404/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:11.276859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65e1e1-f283-4980-ba15-e19ccd73ec71 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:11.277143 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65e1e1-f283-4980-ba15-e19ccd73ec71 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:11.277447 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65e1e1-f283-4980-ba15-e19ccd73ec71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:11.298235 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:11.305523 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-fc65e1e1-f283-4980-ba15-e19ccd73ec71 None None] MFA Rules not processed for user `66510ba7a24246efb73503326ebac404`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:11.306739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-fc65e1e1-f283-4980-ba15-e19ccd73ec71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:11.374540 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1462/2893] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:11.385658 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a61b2-be04-4209-b3d5-4be41da8183f admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:11.387338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-a78a61b2-be04-4209-b3d5-4be41da8183f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:11.410625 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:11.446665 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a61b2-be04-4209-b3d5-4be41da8183f admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:11.450303 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff679bc-a2d2-473d-851c-d795f1ea1652 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:11.450944 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff679bc-a2d2-473d-851c-d795f1ea1652 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:11.454220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff679bc-a2d2-473d-851c-d795f1ea1652 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:11.454220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff679bc-a2d2-473d-851c-d795f1ea1652 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:11.460711 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff679bc-a2d2-473d-851c-d795f1ea1652 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:11.462189 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff679bc-a2d2-473d-851c-d795f1ea1652 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:11.472092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1432/2894] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:11.647819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65e1e1-f283-4980-ba15-e19ccd73ec71 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:11.657949 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65e1e1-f283-4980-ba15-e19ccd73ec71 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:11.661564 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c949ca-1dab-4691-b634-eec07f4e87dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:11.662107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c949ca-1dab-4691-b634-eec07f4e87dd service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:11.662390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c949ca-1dab-4691-b634-eec07f4e87dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:11.662665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c949ca-1dab-4691-b634-eec07f4e87dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:11.717865 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c949ca-1dab-4691-b634-eec07f4e87dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:11.719324 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c949ca-1dab-4691-b634-eec07f4e87dd service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:11.728595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1463/2895] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:21.473131 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=17 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:21.724592 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=57 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:24.250093 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff679bc-a2d2-473d-851c-d795f1ea1652 service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:24.311885 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff679bc-a2d2-473d-851c-d795f1ea1652 service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:24.317015 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5929ef-2761-493f-bd00-54d78c468392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:24.317646 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5929ef-2761-493f-bd00-54d78c468392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:24.317930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5929ef-2761-493f-bd00-54d78c468392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:24.318205 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5929ef-2761-493f-bd00-54d78c468392 admin admin] PATH_INFO: `/v3/users/9d8f221166664e67b40249b3f53aa974` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:24.319341 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5929ef-2761-493f-bd00-54d78c468392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d8f221166664e67b40249b3f53aa974)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:24.321257 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5929ef-2761-493f-bd00-54d78c468392 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:24.497214 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-5e5929ef-2761-493f-bd00-54d78c468392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d8f221166664e67b40249b3f53aa974'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:24.569091 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-5e5929ef-2761-493f-bd00-54d78c468392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d8f221166664e67b40249b3f53aa974'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:24.569704 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-5e5929ef-2761-493f-bd00-54d78c468392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d8f221166664e67b40249b3f53aa974'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:24.582358 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-5e5929ef-2761-493f-bd00-54d78c468392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d8f221166664e67b40249b3f53aa974'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:24.589159 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-5e5929ef-2761-493f-bd00-54d78c468392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d8f221166664e67b40249b3f53aa974'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:24.595028 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1433/2896] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:23:24 2022] DELETE /identity/v3/users/9d8f221166664e67b40249b3f53aa974 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:24.599621 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1a266-8ba3-4f9d-9077-be4991e23726 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:24.600099 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1a266-8ba3-4f9d-9077-be4991e23726 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:24.600193 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1a266-8ba3-4f9d-9077-be4991e23726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:24.619429 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-12d1a266-8ba3-4f9d-9077-be4991e23726 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:24.620623 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-12d1a266-8ba3-4f9d-9077-be4991e23726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:24.678137 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1464/2897] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:24.687810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5929ef-2761-493f-bd00-54d78c468392 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:24.689252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-5e5929ef-2761-493f-bd00-54d78c468392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:24.745096 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5929ef-2761-493f-bd00-54d78c468392 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:24.748783 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48cb707-7c20-4d35-92a6-4c15498471ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:24.749416 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cb707-7c20-4d35-92a6-4c15498471ab service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:24.749692 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cb707-7c20-4d35-92a6-4c15498471ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:24.749963 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cb707-7c20-4d35-92a6-4c15498471ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:24.759284 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48cb707-7c20-4d35-92a6-4c15498471ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:24.760713 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48cb707-7c20-4d35-92a6-4c15498471ab service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:24.770330 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1434/2898] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:25.167397 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1a266-8ba3-4f9d-9077-be4991e23726 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:25.168927 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-12d1a266-8ba3-4f9d-9077-be4991e23726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:25.185178 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1a266-8ba3-4f9d-9077-be4991e23726 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:25.191314 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:25.192218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:25.192386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:25.192665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d admin admin] PATH_INFO: `/v3/projects/8a7f780589ea42a0ab619506a9e1cbb7` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:25.193989 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a7f780589ea42a0ab619506a9e1cbb7)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:25.196152 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:25.302743 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a7f780589ea42a0ab619506a9e1cbb7', 'request_id': 'req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:25.308801 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a7f780589ea42a0ab619506a9e1cbb7', 'request_id': 'req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:25.388694 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a7f780589ea42a0ab619506a9e1cbb7', 'request_id': 'req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:25.399429 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d admin admin] The token cache is being invalidate because project 8a7f780589ea42a0ab619506a9e1cbb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:23:25.399716 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:25.400345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:25.401839 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1465/2899] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:23:25 2022] DELETE /identity/v3/projects/8a7f780589ea42a0ab619506a9e1cbb7 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:25.405780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48cb707-7c20-4d35-92a6-4c15498471ab service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:25.415277 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48cb707-7c20-4d35-92a6-4c15498471ab service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:25.433410 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:25.470669 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf1c53-2187-4849-a3c3-a8856dc797e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:25.471324 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf1c53-2187-4849-a3c3-a8856dc797e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:25.471607 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf1c53-2187-4849-a3c3-a8856dc797e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:25.471885 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf1c53-2187-4849-a3c3-a8856dc797e9 admin admin] PATH_INFO: `/v3/users/66510ba7a24246efb73503326ebac404` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:25.473017 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf1c53-2187-4849-a3c3-a8856dc797e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66510ba7a24246efb73503326ebac404)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:25.474976 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf1c53-2187-4849-a3c3-a8856dc797e9 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:25.630929 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-03bf1c53-2187-4849-a3c3-a8856dc797e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66510ba7a24246efb73503326ebac404'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:25.691240 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-03bf1c53-2187-4849-a3c3-a8856dc797e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66510ba7a24246efb73503326ebac404'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:25.691853 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-03bf1c53-2187-4849-a3c3-a8856dc797e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66510ba7a24246efb73503326ebac404'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:25.704004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-03bf1c53-2187-4849-a3c3-a8856dc797e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66510ba7a24246efb73503326ebac404'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:25.710780 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-03bf1c53-2187-4849-a3c3-a8856dc797e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66510ba7a24246efb73503326ebac404'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:25.716739 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1435/2900] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:23:25 2022] DELETE /identity/v3/users/66510ba7a24246efb73503326ebac404 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:26.097305 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:26.098869 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:26.112129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cd3a1-f3e3-4b79-9790-06c6e7697f6d admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:26.168566 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993cab34-20b7-4cd3-b32b-aca863dc6cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:26.169129 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cab34-20b7-4cd3-b32b-aca863dc6cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:26.169411 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cab34-20b7-4cd3-b32b-aca863dc6cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:26.169686 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cab34-20b7-4cd3-b32b-aca863dc6cc3 admin admin] PATH_INFO: `/v3/projects/cc672b0a96954688aca2a47fabb1de3b` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:26.170810 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993cab34-20b7-4cd3-b32b-aca863dc6cc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc672b0a96954688aca2a47fabb1de3b)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:26.172707 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993cab34-20b7-4cd3-b32b-aca863dc6cc3 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:26.250697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:26.267017 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-993cab34-20b7-4cd3-b32b-aca863dc6cc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc672b0a96954688aca2a47fabb1de3b', 'request_id': 'req-993cab34-20b7-4cd3-b32b-aca863dc6cc3'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:26.273657 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-993cab34-20b7-4cd3-b32b-aca863dc6cc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc672b0a96954688aca2a47fabb1de3b', 'request_id': 'req-993cab34-20b7-4cd3-b32b-aca863dc6cc3'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:26.357571 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-993cab34-20b7-4cd3-b32b-aca863dc6cc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc672b0a96954688aca2a47fabb1de3b', 'request_id': 'req-993cab34-20b7-4cd3-b32b-aca863dc6cc3'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:26.367732 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-993cab34-20b7-4cd3-b32b-aca863dc6cc3 admin admin] The token cache is being invalidate because project cc672b0a96954688aca2a47fabb1de3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:23:26.368018 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-993cab34-20b7-4cd3-b32b-aca863dc6cc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:26.368625 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-993cab34-20b7-4cd3-b32b-aca863dc6cc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:26.370086 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1466/2901] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:23:26 2022] DELETE /identity/v3/projects/cc672b0a96954688aca2a47fabb1de3b => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:26.375984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11a0da-2f61-447e-bbff-43636545682f None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:26.376271 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11a0da-2f61-447e-bbff-43636545682f None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:26.380409 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11a0da-2f61-447e-bbff-43636545682f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:26.396883 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-5e11a0da-2f61-447e-bbff-43636545682f None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:26.398035 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-5e11a0da-2f61-447e-bbff-43636545682f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:26.454617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1436/2902] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:26.462807 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfde814-4683-448d-81eb-46f195eb7d38 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:26.463107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfde814-4683-448d-81eb-46f195eb7d38 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:26.463381 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfde814-4683-448d-81eb-46f195eb7d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:26.482904 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-5bfde814-4683-448d-81eb-46f195eb7d38 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:26.484092 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5bfde814-4683-448d-81eb-46f195eb7d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:26.491742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1467/2903] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:26.495926 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11a0da-2f61-447e-bbff-43636545682f None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:26.505987 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11a0da-2f61-447e-bbff-43636545682f None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:26.510461 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dec91a-db7e-44a7-93ce-3eddfe4fc236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:26.511088 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dec91a-db7e-44a7-93ce-3eddfe4fc236 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:26.511368 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dec91a-db7e-44a7-93ce-3eddfe4fc236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:26.511641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dec91a-db7e-44a7-93ce-3eddfe4fc236 admin admin] PATH_INFO: `/v3/domains` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:26.512307 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dec91a-db7e-44a7-93ce-3eddfe4fc236 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81594) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:23:26.512608 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dec91a-db7e-44a7-93ce-3eddfe4fc236 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:26.514459 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dec91a-db7e-44a7-93ce-3eddfe4fc236 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:26.526264 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1437/2904] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 15:23:26.530385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfde814-4683-448d-81eb-46f195eb7d38 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:26.539452 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfde814-4683-448d-81eb-46f195eb7d38 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:26.544065 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7ba22-da7b-46da-858b-0a43f32973a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:26.544646 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7ba22-da7b-46da-858b-0a43f32973a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:26.544923 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7ba22-da7b-46da-858b-0a43f32973a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:26.545192 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7ba22-da7b-46da-858b-0a43f32973a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:26.545810 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7ba22-da7b-46da-858b-0a43f32973a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:26.547726 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7ba22-da7b-46da-858b-0a43f32973a9 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:26.661800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1468/2905] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:23:26 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:23:26.665900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dec91a-db7e-44a7-93ce-3eddfe4fc236 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:26.675880 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dec91a-db7e-44a7-93ce-3eddfe4fc236 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:26.702117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:26.733329 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b3a6b-1ed1-4c20-979e-d116455ea11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:26.733935 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b3a6b-1ed1-4c20-979e-d116455ea11b admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:26.734589 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b3a6b-1ed1-4c20-979e-d116455ea11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:26.735117 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b3a6b-1ed1-4c20-979e-d116455ea11b admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:26.736114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b3a6b-1ed1-4c20-979e-d116455ea11b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:26.738114 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b3a6b-1ed1-4c20-979e-d116455ea11b admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:26.832684 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1438/2906] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:23:26 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:23:26.836764 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7ba22-da7b-46da-858b-0a43f32973a9 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:26.847538 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7ba22-da7b-46da-858b-0a43f32973a9 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:26.852181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2fe520-b4fe-4c72-b3df-cc4aa9a5d367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:26.852773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fe520-b4fe-4c72-b3df-cc4aa9a5d367 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:26.853058 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fe520-b4fe-4c72-b3df-cc4aa9a5d367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:26.855791 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fe520-b4fe-4c72-b3df-cc4aa9a5d367 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:26.855791 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2fe520-b4fe-4c72-b3df-cc4aa9a5d367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:26.855791 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2fe520-b4fe-4c72-b3df-cc4aa9a5d367 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:26.865261 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1469/2907] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:23:26.869228 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b3a6b-1ed1-4c20-979e-d116455ea11b admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:26.879039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b3a6b-1ed1-4c20-979e-d116455ea11b admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:26.883556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434a09c-9f8b-4b51-ba55-744d2692e3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:26.884143 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434a09c-9f8b-4b51-ba55-744d2692e3df admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:26.884420 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434a09c-9f8b-4b51-ba55-744d2692e3df admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:26.884690 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434a09c-9f8b-4b51-ba55-744d2692e3df admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:26.885319 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434a09c-9f8b-4b51-ba55-744d2692e3df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:26.886944 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434a09c-9f8b-4b51-ba55-744d2692e3df admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:26.897517 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1439/2908] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:23:26.901567 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2fe520-b4fe-4c72-b3df-cc4aa9a5d367 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:26.912013 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2fe520-b4fe-4c72-b3df-cc4aa9a5d367 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:26.916613 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d07d5-a98a-4341-b031-c873762d5f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:26.917253 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d07d5-a98a-4341-b031-c873762d5f77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:26.917534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d07d5-a98a-4341-b031-c873762d5f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:26.917808 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d07d5-a98a-4341-b031-c873762d5f77 admin admin] PATH_INFO: `/v3/projects/6eadeb10f5c141159e54a4dfda7c34cb/users/b15d3636d8244a42a22d34f5c90d3d52/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:26.931868 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d07d5-a98a-4341-b031-c873762d5f77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eadeb10f5c141159e54a4dfda7c34cb, user_id=b15d3636d8244a42a22d34f5c90d3d52, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:26.934496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d07d5-a98a-4341-b031-c873762d5f77 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:27.005800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1470/2909] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:23:26 2022] PUT /identity/v3/projects/6eadeb10f5c141159e54a4dfda7c34cb/users/b15d3636d8244a42a22d34f5c90d3d52/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:27.017210 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd090f1-139b-437a-b5b2-b86caa3dae05 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:27.017497 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd090f1-139b-437a-b5b2-b86caa3dae05 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:27.018285 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd090f1-139b-437a-b5b2-b86caa3dae05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:27.046221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-8dd090f1-139b-437a-b5b2-b86caa3dae05 None None] MFA Rules not processed for user `b15d3636d8244a42a22d34f5c90d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:27.047456 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8dd090f1-139b-437a-b5b2-b86caa3dae05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:27.110918 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1440/2910] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:27.115391 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d07d5-a98a-4341-b031-c873762d5f77 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:27.180140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d07d5-a98a-4341-b031-c873762d5f77 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:27.184797 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498e392-48a0-4d7f-afdd-852b04610bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:27.185396 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498e392-48a0-4d7f-afdd-852b04610bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:27.185686 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498e392-48a0-4d7f-afdd-852b04610bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:27.185951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498e392-48a0-4d7f-afdd-852b04610bb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:27.186569 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498e392-48a0-4d7f-afdd-852b04610bb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:27.188507 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498e392-48a0-4d7f-afdd-852b04610bb8 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:27.281678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1471/2911] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:23:27 2022] POST /identity/v3/projects => generated 387 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:23:27.285652 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd090f1-139b-437a-b5b2-b86caa3dae05 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:27.296221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd090f1-139b-437a-b5b2-b86caa3dae05 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:27.350705 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fbb8c-942a-4979-8b8d-70d10b2b62f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:27.351327 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fbb8c-942a-4979-8b8d-70d10b2b62f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:27.351608 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fbb8c-942a-4979-8b8d-70d10b2b62f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:27.351882 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fbb8c-942a-4979-8b8d-70d10b2b62f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:27.352499 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fbb8c-942a-4979-8b8d-70d10b2b62f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:27.354422 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fbb8c-942a-4979-8b8d-70d10b2b62f9 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:27.422738 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1441/2912] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:23:27 2022] POST /identity/v3/users => generated 300 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:23:27.426728 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498e392-48a0-4d7f-afdd-852b04610bb8 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:27.437174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498e392-48a0-4d7f-afdd-852b04610bb8 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:27.442273 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d693367-3de6-482d-8a7e-e2bdb51b2ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:27.443001 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d693367-3de6-482d-8a7e-e2bdb51b2ddc admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:27.443281 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d693367-3de6-482d-8a7e-e2bdb51b2ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:27.443550 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d693367-3de6-482d-8a7e-e2bdb51b2ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:27.444176 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d693367-3de6-482d-8a7e-e2bdb51b2ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:27.445752 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d693367-3de6-482d-8a7e-e2bdb51b2ddc admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:27.456653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1472/2913] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:23:27.460695 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fbb8c-942a-4979-8b8d-70d10b2b62f9 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:27.470515 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fbb8c-942a-4979-8b8d-70d10b2b62f9 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:27.475148 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12faa4-c947-40b3-9915-1fc80d2d1ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:27.475792 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12faa4-c947-40b3-9915-1fc80d2d1ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:27.476074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12faa4-c947-40b3-9915-1fc80d2d1ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:27.476348 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12faa4-c947-40b3-9915-1fc80d2d1ffb admin admin] PATH_INFO: `/v3/projects/fdf557f454794689a38b0686a80d84fa/users/158046100c8846409082a994a47abcc0/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:27.490480 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12faa4-c947-40b3-9915-1fc80d2d1ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf557f454794689a38b0686a80d84fa, user_id=158046100c8846409082a994a47abcc0, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:27.493153 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12faa4-c947-40b3-9915-1fc80d2d1ffb admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:27.558250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1442/2914] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:23:27 2022] PUT /identity/v3/projects/fdf557f454794689a38b0686a80d84fa/users/158046100c8846409082a994a47abcc0/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:23:27.562436 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d693367-3de6-482d-8a7e-e2bdb51b2ddc admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:27.573599 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d693367-3de6-482d-8a7e-e2bdb51b2ddc admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:27.594233 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:27.629520 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db04a4-9c02-4022-a3b6-2265762dae4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:27.630122 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db04a4-9c02-4022-a3b6-2265762dae4f admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:27.630400 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db04a4-9c02-4022-a3b6-2265762dae4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:27.630668 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db04a4-9c02-4022-a3b6-2265762dae4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:27.631326 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db04a4-9c02-4022-a3b6-2265762dae4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:27.632974 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db04a4-9c02-4022-a3b6-2265762dae4f admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:27.642689 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1473/2915] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:23:27.646950 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12faa4-c947-40b3-9915-1fc80d2d1ffb admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:27.652338 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:27.657553 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12faa4-c947-40b3-9915-1fc80d2d1ffb admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:27.662077 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af070b-68c8-43e6-99ab-90ca7e0c1cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:27.662717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af070b-68c8-43e6-99ab-90ca7e0c1cb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:27.666854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af070b-68c8-43e6-99ab-90ca7e0c1cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:27.666854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af070b-68c8-43e6-99ab-90ca7e0c1cb8 admin admin] PATH_INFO: `/v3/projects/fdf557f454794689a38b0686a80d84fa/users/158046100c8846409082a994a47abcc0/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:27.666854 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af070b-68c8-43e6-99ab-90ca7e0c1cb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf557f454794689a38b0686a80d84fa, user_id=158046100c8846409082a994a47abcc0, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:27.667839 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af070b-68c8-43e6-99ab-90ca7e0c1cb8 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:27.678389 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1443/2916] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:23:27 2022] PUT /identity/v3/projects/fdf557f454794689a38b0686a80d84fa/users/158046100c8846409082a994a47abcc0/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:27.688495 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234aee52-199f-4ad8-a681-ddce56fe69ab None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:27.688804 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234aee52-199f-4ad8-a681-ddce56fe69ab None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:27.689076 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234aee52-199f-4ad8-a681-ddce56fe69ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:27.717327 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-234aee52-199f-4ad8-a681-ddce56fe69ab None None] MFA Rules not processed for user `158046100c8846409082a994a47abcc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:27.718553 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-234aee52-199f-4ad8-a681-ddce56fe69ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:27.785147 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1474/2917] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 15:23:27.796914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af070b-68c8-43e6-99ab-90ca7e0c1cb8 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:27.798590 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c9af070b-68c8-43e6-99ab-90ca7e0c1cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:27.855948 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af070b-68c8-43e6-99ab-90ca7e0c1cb8 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:27.859576 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54c004-cf1e-4dc6-a5ea-9633fe17c139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:27.860183 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54c004-cf1e-4dc6-a5ea-9633fe17c139 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:27.860462 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54c004-cf1e-4dc6-a5ea-9633fe17c139 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:27.860735 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54c004-cf1e-4dc6-a5ea-9633fe17c139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:27.870317 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54c004-cf1e-4dc6-a5ea-9633fe17c139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:27.871903 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54c004-cf1e-4dc6-a5ea-9633fe17c139 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:27.881811 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1444/2918] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:27.946171 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234aee52-199f-4ad8-a681-ddce56fe69ab None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:28.007366 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234aee52-199f-4ad8-a681-ddce56fe69ab None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:28.011920 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcde086-06d7-4a00-9f2c-4009e5ac46da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:28.012554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcde086-06d7-4a00-9f2c-4009e5ac46da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:28.012836 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcde086-06d7-4a00-9f2c-4009e5ac46da admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:28.013112 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcde086-06d7-4a00-9f2c-4009e5ac46da admin admin] PATH_INFO: `/v3/users/b15d3636d8244a42a22d34f5c90d3d52` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:28.014497 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcde086-06d7-4a00-9f2c-4009e5ac46da admin admin] RBAC: Authorizing `identity:delete_user(user_id=b15d3636d8244a42a22d34f5c90d3d52)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:28.016434 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcde086-06d7-4a00-9f2c-4009e5ac46da admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:28.213378 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-cdcde086-06d7-4a00-9f2c-4009e5ac46da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b15d3636d8244a42a22d34f5c90d3d52'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:28.215089 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:28.291873 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-cdcde086-06d7-4a00-9f2c-4009e5ac46da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b15d3636d8244a42a22d34f5c90d3d52'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:28.305289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-cdcde086-06d7-4a00-9f2c-4009e5ac46da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b15d3636d8244a42a22d34f5c90d3d52'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:28.305942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-cdcde086-06d7-4a00-9f2c-4009e5ac46da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b15d3636d8244a42a22d34f5c90d3d52'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:28.312721 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-cdcde086-06d7-4a00-9f2c-4009e5ac46da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b15d3636d8244a42a22d34f5c90d3d52'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:28.318536 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1475/2919] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:23:27 2022] DELETE /identity/v3/users/b15d3636d8244a42a22d34f5c90d3d52 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:28.323255 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfceed4-4683-4356-85bc-8c1e81d7fcbb None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:28.323636 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfceed4-4683-4356-85bc-8c1e81d7fcbb None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:28.323914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfceed4-4683-4356-85bc-8c1e81d7fcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:28.344027 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-bcfceed4-4683-4356-85bc-8c1e81d7fcbb None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:28.345210 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-bcfceed4-4683-4356-85bc-8c1e81d7fcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:28.401976 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1445/2920] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:28.412086 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcde086-06d7-4a00-9f2c-4009e5ac46da admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:28.413512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-cdcde086-06d7-4a00-9f2c-4009e5ac46da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:28.473389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcde086-06d7-4a00-9f2c-4009e5ac46da admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:28.477251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dca456-2dd4-4762-9bb9-3e699e3d9a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:28.477887 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dca456-2dd4-4762-9bb9-3e699e3d9a35 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:28.478169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dca456-2dd4-4762-9bb9-3e699e3d9a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:28.478456 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dca456-2dd4-4762-9bb9-3e699e3d9a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:28.487890 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dca456-2dd4-4762-9bb9-3e699e3d9a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:28.489429 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dca456-2dd4-4762-9bb9-3e699e3d9a35 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:28.499150 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1476/2921] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:28.875921 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfceed4-4683-4356-85bc-8c1e81d7fcbb None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:28.877379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-bcfceed4-4683-4356-85bc-8c1e81d7fcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:28.892914 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfceed4-4683-4356-85bc-8c1e81d7fcbb None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:28.899570 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5352b-3449-485b-bc47-4a72748bd910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:28.900215 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5352b-3449-485b-bc47-4a72748bd910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:28.900503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5352b-3449-485b-bc47-4a72748bd910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:28.900781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5352b-3449-485b-bc47-4a72748bd910 admin admin] PATH_INFO: `/v3/projects/6eadeb10f5c141159e54a4dfda7c34cb` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:28.901961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da5352b-3449-485b-bc47-4a72748bd910 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eadeb10f5c141159e54a4dfda7c34cb)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:28.903907 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da5352b-3449-485b-bc47-4a72748bd910 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:29.005632 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-7da5352b-3449-485b-bc47-4a72748bd910 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eadeb10f5c141159e54a4dfda7c34cb', 'request_id': 'req-7da5352b-3449-485b-bc47-4a72748bd910'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:29.011143 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-7da5352b-3449-485b-bc47-4a72748bd910 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eadeb10f5c141159e54a4dfda7c34cb', 'request_id': 'req-7da5352b-3449-485b-bc47-4a72748bd910'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:29.077557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-7da5352b-3449-485b-bc47-4a72748bd910 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eadeb10f5c141159e54a4dfda7c34cb', 'request_id': 'req-7da5352b-3449-485b-bc47-4a72748bd910'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:29.088950 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-7da5352b-3449-485b-bc47-4a72748bd910 admin admin] The token cache is being invalidate because project 6eadeb10f5c141159e54a4dfda7c34cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:23:29.089243 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-7da5352b-3449-485b-bc47-4a72748bd910 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:29.089875 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-7da5352b-3449-485b-bc47-4a72748bd910 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:29.091613 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1446/2922] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:23:28 2022] DELETE /identity/v3/projects/6eadeb10f5c141159e54a4dfda7c34cb => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:23:29.095979 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dca456-2dd4-4762-9bb9-3e699e3d9a35 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:29.106325 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dca456-2dd4-4762-9bb9-3e699e3d9a35 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:29.162298 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147b606-dfdc-484d-88fd-fef21fd5b467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:29.162942 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147b606-dfdc-484d-88fd-fef21fd5b467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:29.163222 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147b606-dfdc-484d-88fd-fef21fd5b467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:29.163491 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147b606-dfdc-484d-88fd-fef21fd5b467 admin admin] PATH_INFO: `/v3/users/158046100c8846409082a994a47abcc0` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:29.164669 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147b606-dfdc-484d-88fd-fef21fd5b467 admin admin] RBAC: Authorizing `identity:delete_user(user_id=158046100c8846409082a994a47abcc0)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:29.166593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147b606-dfdc-484d-88fd-fef21fd5b467 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:29.383102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-1147b606-dfdc-484d-88fd-fef21fd5b467 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '158046100c8846409082a994a47abcc0'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:29.449638 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-1147b606-dfdc-484d-88fd-fef21fd5b467 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '158046100c8846409082a994a47abcc0'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:29.462652 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:29.463965 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-1147b606-dfdc-484d-88fd-fef21fd5b467 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '158046100c8846409082a994a47abcc0'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:29.464641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-1147b606-dfdc-484d-88fd-fef21fd5b467 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '158046100c8846409082a994a47abcc0'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:29.471911 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-1147b606-dfdc-484d-88fd-fef21fd5b467 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '158046100c8846409082a994a47abcc0'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:29.478665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1477/2923] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:23:29 2022] DELETE /identity/v3/users/158046100c8846409082a994a47abcc0 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:29.915023 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5352b-3449-485b-bc47-4a72748bd910 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:29.916638 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-7da5352b-3449-485b-bc47-4a72748bd910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:29.930221 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5352b-3449-485b-bc47-4a72748bd910 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:29.942669 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:29.991773 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997507cb-a1f8-4cb9-b860-ca0fa4549387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:29.992403 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997507cb-a1f8-4cb9-b860-ca0fa4549387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:29.992781 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997507cb-a1f8-4cb9-b860-ca0fa4549387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:29.993061 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997507cb-a1f8-4cb9-b860-ca0fa4549387 admin admin] PATH_INFO: `/v3/projects/fdf557f454794689a38b0686a80d84fa` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:29.994168 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997507cb-a1f8-4cb9-b860-ca0fa4549387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdf557f454794689a38b0686a80d84fa)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:29.996105 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997507cb-a1f8-4cb9-b860-ca0fa4549387 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:30.123597 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-997507cb-a1f8-4cb9-b860-ca0fa4549387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdf557f454794689a38b0686a80d84fa', 'request_id': 'req-997507cb-a1f8-4cb9-b860-ca0fa4549387'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:30.128834 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-997507cb-a1f8-4cb9-b860-ca0fa4549387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdf557f454794689a38b0686a80d84fa', 'request_id': 'req-997507cb-a1f8-4cb9-b860-ca0fa4549387'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:30.194125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-997507cb-a1f8-4cb9-b860-ca0fa4549387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdf557f454794689a38b0686a80d84fa', 'request_id': 'req-997507cb-a1f8-4cb9-b860-ca0fa4549387'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:30.205599 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-997507cb-a1f8-4cb9-b860-ca0fa4549387 admin admin] The token cache is being invalidate because project fdf557f454794689a38b0686a80d84fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:23:30.205888 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-997507cb-a1f8-4cb9-b860-ca0fa4549387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:30.206519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-997507cb-a1f8-4cb9-b860-ca0fa4549387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:30.208589 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1447/2924] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:23:29 2022] DELETE /identity/v3/projects/fdf557f454794689a38b0686a80d84fa => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:30.213562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b8495-61e7-415a-a807-b67c19c98495 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:30.213849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b8495-61e7-415a-a807-b67c19c98495 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:30.214123 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b8495-61e7-415a-a807-b67c19c98495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:30.233386 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-070b8495-61e7-415a-a807-b67c19c98495 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:30.234581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-070b8495-61e7-415a-a807-b67c19c98495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:30.292367 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1478/2925] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:30.300147 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193bea2-34e6-48ab-9d14-e3ee44f03782 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:30.300438 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193bea2-34e6-48ab-9d14-e3ee44f03782 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:30.300714 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193bea2-34e6-48ab-9d14-e3ee44f03782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:30.321432 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-f193bea2-34e6-48ab-9d14-e3ee44f03782 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:30.322615 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-f193bea2-34e6-48ab-9d14-e3ee44f03782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:30.330378 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1448/2926] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 15:23:30.335047 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b8495-61e7-415a-a807-b67c19c98495 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:30.344459 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b8495-61e7-415a-a807-b67c19c98495 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:30.349070 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cef641-5393-4e40-a31f-9148c67e54b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:30.349659 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cef641-5393-4e40-a31f-9148c67e54b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:30.349941 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cef641-5393-4e40-a31f-9148c67e54b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:30.350215 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cef641-5393-4e40-a31f-9148c67e54b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:30.350895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cef641-5393-4e40-a31f-9148c67e54b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:23:30.351202 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cef641-5393-4e40-a31f-9148c67e54b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:30.352781 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cef641-5393-4e40-a31f-9148c67e54b8 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:30.364061 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1479/2927] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 15:23:30.368199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193bea2-34e6-48ab-9d14-e3ee44f03782 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:30.377057 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193bea2-34e6-48ab-9d14-e3ee44f03782 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:30.382038 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9240397-ba43-4a14-8051-9abe6a34b59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:30.382623 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9240397-ba43-4a14-8051-9abe6a34b59d admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:30.382939 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9240397-ba43-4a14-8051-9abe6a34b59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:30.383232 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9240397-ba43-4a14-8051-9abe6a34b59d admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:30.383824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9240397-ba43-4a14-8051-9abe6a34b59d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:30.385699 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9240397-ba43-4a14-8051-9abe6a34b59d admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:30.498510 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1449/2928] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:23:30 2022] POST /identity/v3/projects => generated 399 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:23:30.502740 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cef641-5393-4e40-a31f-9148c67e54b8 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:30.513406 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cef641-5393-4e40-a31f-9148c67e54b8 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:30.569845 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927af7e-9f51-463c-92f0-034f46870720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:30.570439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927af7e-9f51-463c-92f0-034f46870720 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:30.570717 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927af7e-9f51-463c-92f0-034f46870720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:30.571021 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927af7e-9f51-463c-92f0-034f46870720 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:30.571672 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927af7e-9f51-463c-92f0-034f46870720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:30.573584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927af7e-9f51-463c-92f0-034f46870720 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:30.697552 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1480/2929] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:23:30 2022] POST /identity/v3/users => generated 300 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:23:30.701712 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9240397-ba43-4a14-8051-9abe6a34b59d admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:30.711995 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9240397-ba43-4a14-8051-9abe6a34b59d admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:30.716592 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bce7c5-afc1-4dfc-b698-96d95cbf65de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:30.717189 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bce7c5-afc1-4dfc-b698-96d95cbf65de admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:30.717470 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bce7c5-afc1-4dfc-b698-96d95cbf65de admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:30.717743 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bce7c5-afc1-4dfc-b698-96d95cbf65de admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:30.718384 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bce7c5-afc1-4dfc-b698-96d95cbf65de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:30.720001 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bce7c5-afc1-4dfc-b698-96d95cbf65de admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:30.730278 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1450/2930] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:23:30.734069 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927af7e-9f51-463c-92f0-034f46870720 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:30.744358 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927af7e-9f51-463c-92f0-034f46870720 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:30.748950 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4286e76-2e2b-4578-ae51-6be1fba5f30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:30.749477 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4286e76-2e2b-4578-ae51-6be1fba5f30f admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:30.749754 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4286e76-2e2b-4578-ae51-6be1fba5f30f admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:30.750026 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4286e76-2e2b-4578-ae51-6be1fba5f30f admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:30.750655 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4286e76-2e2b-4578-ae51-6be1fba5f30f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:30.752284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4286e76-2e2b-4578-ae51-6be1fba5f30f admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:30.763203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1481/2931] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 15:23:30.767281 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bce7c5-afc1-4dfc-b698-96d95cbf65de admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:30.776862 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bce7c5-afc1-4dfc-b698-96d95cbf65de admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:30.781360 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5162a-dc9a-4fd7-b549-b55a451b5bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:30.781994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5162a-dc9a-4fd7-b549-b55a451b5bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:30.782273 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5162a-dc9a-4fd7-b549-b55a451b5bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:30.782545 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5162a-dc9a-4fd7-b549-b55a451b5bfc admin admin] PATH_INFO: `/v3/projects/11615d7cea1f4a878c5d5be1c78d735f/users/33c7b2a8b9654bde9c7508993f9d295a/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:30.797111 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5162a-dc9a-4fd7-b549-b55a451b5bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=11615d7cea1f4a878c5d5be1c78d735f, user_id=33c7b2a8b9654bde9c7508993f9d295a, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:30.799908 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5162a-dc9a-4fd7-b549-b55a451b5bfc admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:30.868640 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1451/2932] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:23:30 2022] PUT /identity/v3/projects/11615d7cea1f4a878c5d5be1c78d735f/users/33c7b2a8b9654bde9c7508993f9d295a/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:23:30.880343 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cdcdb-0dff-489f-bf60-0e82e2162dc9 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:30.880624 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cdcdb-0dff-489f-bf60-0e82e2162dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:30.880895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cdcdb-0dff-489f-bf60-0e82e2162dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:30.907938 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-d57cdcdb-0dff-489f-bf60-0e82e2162dc9 None None] MFA Rules not processed for user `33c7b2a8b9654bde9c7508993f9d295a`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:30.918893 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d57cdcdb-0dff-489f-bf60-0e82e2162dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:30.949554 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:30.971439 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1482/2933] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:30 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:30.976408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5162a-dc9a-4fd7-b549-b55a451b5bfc admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:31.022067 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:31.039129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5162a-dc9a-4fd7-b549-b55a451b5bfc admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:31.043748 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3753a6-3f25-4e7f-b409-11b6a18f58e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:31.044340 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3753a6-3f25-4e7f-b409-11b6a18f58e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:31.044620 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3753a6-3f25-4e7f-b409-11b6a18f58e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:31.044891 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3753a6-3f25-4e7f-b409-11b6a18f58e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:31.045515 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3753a6-3f25-4e7f-b409-11b6a18f58e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:31.047448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3753a6-3f25-4e7f-b409-11b6a18f58e8 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:31.176718 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1452/2934] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:23:30 2022] POST /identity/v3/projects => generated 399 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:23:31.180886 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cdcdb-0dff-489f-bf60-0e82e2162dc9 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:31.191760 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cdcdb-0dff-489f-bf60-0e82e2162dc9 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:31.247563 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40545704-5a37-4be0-bfa4-fa593e89ca8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:31.251995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40545704-5a37-4be0-bfa4-fa593e89ca8c admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:31.251995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40545704-5a37-4be0-bfa4-fa593e89ca8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:31.251995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40545704-5a37-4be0-bfa4-fa593e89ca8c admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:31.251995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40545704-5a37-4be0-bfa4-fa593e89ca8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:31.251995 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40545704-5a37-4be0-bfa4-fa593e89ca8c admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:31.340650 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1483/2935] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:23:31 2022] POST /identity/v3/users => generated 306 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:23:31.344544 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3753a6-3f25-4e7f-b409-11b6a18f58e8 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:31.355424 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3753a6-3f25-4e7f-b409-11b6a18f58e8 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:31.360250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b13b4f-cce5-425a-865d-e0faeac0d8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:31.360860 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b13b4f-cce5-425a-865d-e0faeac0d8ab admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:31.361141 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b13b4f-cce5-425a-865d-e0faeac0d8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:31.361413 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b13b4f-cce5-425a-865d-e0faeac0d8ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:31.362066 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b13b4f-cce5-425a-865d-e0faeac0d8ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:31.363768 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b13b4f-cce5-425a-865d-e0faeac0d8ab admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:31.373657 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1453/2936] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:23:31.377545 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40545704-5a37-4be0-bfa4-fa593e89ca8c admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:31.387411 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40545704-5a37-4be0-bfa4-fa593e89ca8c admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:31.392219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaabbda-4d74-4dcd-b55b-e3a682d7fd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:31.392882 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaabbda-4d74-4dcd-b55b-e3a682d7fd0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:31.393158 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaabbda-4d74-4dcd-b55b-e3a682d7fd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:31.393429 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaabbda-4d74-4dcd-b55b-e3a682d7fd0e admin admin] PATH_INFO: `/v3/projects/f97dc89cbdd64289a44f7a6dc3b601a0/users/1b9e2fb7245141328c717eec22279a7e/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:31.408389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaabbda-4d74-4dcd-b55b-e3a682d7fd0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f97dc89cbdd64289a44f7a6dc3b601a0, user_id=1b9e2fb7245141328c717eec22279a7e, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:31.412704 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaabbda-4d74-4dcd-b55b-e3a682d7fd0e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:31.509037 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1484/2937] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:23:31 2022] PUT /identity/v3/projects/f97dc89cbdd64289a44f7a6dc3b601a0/users/1b9e2fb7245141328c717eec22279a7e/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:23:31.513092 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b13b4f-cce5-425a-865d-e0faeac0d8ab admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:31.524199 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b13b4f-cce5-425a-865d-e0faeac0d8ab admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:31.579690 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c0241-c6f1-4b7d-8beb-ff1ee23a2f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:31.580300 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c0241-c6f1-4b7d-8beb-ff1ee23a2f07 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:31.580584 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c0241-c6f1-4b7d-8beb-ff1ee23a2f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:31.580860 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c0241-c6f1-4b7d-8beb-ff1ee23a2f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:31.581994 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5c0241-c6f1-4b7d-8beb-ff1ee23a2f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:31.583627 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5c0241-c6f1-4b7d-8beb-ff1ee23a2f07 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:31.593213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1454/2938] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:23:31.597947 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaabbda-4d74-4dcd-b55b-e3a682d7fd0e admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:31.608385 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaabbda-4d74-4dcd-b55b-e3a682d7fd0e admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:31.613218 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae1f12-ed18-401b-9640-7337eedbf1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:31.613863 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae1f12-ed18-401b-9640-7337eedbf1c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:31.614148 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae1f12-ed18-401b-9640-7337eedbf1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:31.614429 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae1f12-ed18-401b-9640-7337eedbf1c9 admin admin] PATH_INFO: `/v3/projects/f97dc89cbdd64289a44f7a6dc3b601a0/users/1b9e2fb7245141328c717eec22279a7e/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:31.616399 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae1f12-ed18-401b-9640-7337eedbf1c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f97dc89cbdd64289a44f7a6dc3b601a0, user_id=1b9e2fb7245141328c717eec22279a7e, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:31.619069 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae1f12-ed18-401b-9640-7337eedbf1c9 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:31.629089 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1485/2939] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:23:31 2022] PUT /identity/v3/projects/f97dc89cbdd64289a44f7a6dc3b601a0/users/1b9e2fb7245141328c717eec22279a7e/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:31.639376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8571c-cd14-4a0f-a2c1-d6ce6171e271 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:31.639668 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8571c-cd14-4a0f-a2c1-d6ce6171e271 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:31.639949 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8571c-cd14-4a0f-a2c1-d6ce6171e271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:31.668104 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-c9c8571c-cd14-4a0f-a2c1-d6ce6171e271 None None] MFA Rules not processed for user `1b9e2fb7245141328c717eec22279a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:31.669302 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c9c8571c-cd14-4a0f-a2c1-d6ce6171e271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:31.736900 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1455/2940] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:31 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:31.747719 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae1f12-ed18-401b-9640-7337eedbf1c9 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:31.749389 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5fae1f12-ed18-401b-9640-7337eedbf1c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:31.806674 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae1f12-ed18-401b-9640-7337eedbf1c9 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:31.810333 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ed1b2-aec1-49ec-8139-9cbc074cc2ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:31.810964 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ed1b2-aec1-49ec-8139-9cbc074cc2ec service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:31.811251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ed1b2-aec1-49ec-8139-9cbc074cc2ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:31.811529 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ed1b2-aec1-49ec-8139-9cbc074cc2ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:31.820885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ed1b2-aec1-49ec-8139-9cbc074cc2ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:31.822308 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ed1b2-aec1-49ec-8139-9cbc074cc2ec service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:31.832234 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1486/2941] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:31 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:32.059041 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8571c-cd14-4a0f-a2c1-d6ce6171e271 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:32.069549 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8571c-cd14-4a0f-a2c1-d6ce6171e271 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:32.073190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cff78-4eb2-493d-9ef2-a70526c30272 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:32.073791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cff78-4eb2-493d-9ef2-a70526c30272 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:32.074074 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cff78-4eb2-493d-9ef2-a70526c30272 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:32.074347 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cff78-4eb2-493d-9ef2-a70526c30272 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:32.107283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:32.130350 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686cff78-4eb2-493d-9ef2-a70526c30272 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:32.131837 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686cff78-4eb2-493d-9ef2-a70526c30272 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:32.140722 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1456/2942] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:32 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:37.201103 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ed1b2-aec1-49ec-8139-9cbc074cc2ec service cinder] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:37.242860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:37.263562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ed1b2-aec1-49ec-8139-9cbc074cc2ec service cinder] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:37.268420 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f93c60-a623-4c5c-ad86-12a82cfb7223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:37.269690 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f93c60-a623-4c5c-ad86-12a82cfb7223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:37.269978 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f93c60-a623-4c5c-ad86-12a82cfb7223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:37.270249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f93c60-a623-4c5c-ad86-12a82cfb7223 admin admin] PATH_INFO: `/v3/users/33c7b2a8b9654bde9c7508993f9d295a` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:37.271591 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f93c60-a623-4c5c-ad86-12a82cfb7223 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33c7b2a8b9654bde9c7508993f9d295a)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:37.274174 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f93c60-a623-4c5c-ad86-12a82cfb7223 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:37.445423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-31f93c60-a623-4c5c-ad86-12a82cfb7223 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33c7b2a8b9654bde9c7508993f9d295a'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:37.520227 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-31f93c60-a623-4c5c-ad86-12a82cfb7223 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33c7b2a8b9654bde9c7508993f9d295a'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:37.532201 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-31f93c60-a623-4c5c-ad86-12a82cfb7223 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33c7b2a8b9654bde9c7508993f9d295a'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:37.532878 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-31f93c60-a623-4c5c-ad86-12a82cfb7223 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33c7b2a8b9654bde9c7508993f9d295a'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:37.540042 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-31f93c60-a623-4c5c-ad86-12a82cfb7223 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33c7b2a8b9654bde9c7508993f9d295a'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:37.546014 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1487/2943] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:23:37 2022] DELETE /identity/v3/users/33c7b2a8b9654bde9c7508993f9d295a => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:37.550561 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207edeff-3957-4285-9aa4-7387558da826 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:37.551058 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207edeff-3957-4285-9aa4-7387558da826 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:37.552512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207edeff-3957-4285-9aa4-7387558da826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:37.570982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-207edeff-3957-4285-9aa4-7387558da826 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:37.572210 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-207edeff-3957-4285-9aa4-7387558da826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:37.628346 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1457/2944] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:37.638375 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f93c60-a623-4c5c-ad86-12a82cfb7223 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:37.640045 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-31f93c60-a623-4c5c-ad86-12a82cfb7223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:37.695585 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f93c60-a623-4c5c-ad86-12a82cfb7223 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:37.699087 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa715ec-3610-44f2-88cb-8272e2f01dae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:37.699680 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa715ec-3610-44f2-88cb-8272e2f01dae service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:37.699957 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa715ec-3610-44f2-88cb-8272e2f01dae service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:37.700226 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa715ec-3610-44f2-88cb-8272e2f01dae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:37.709258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa715ec-3610-44f2-88cb-8272e2f01dae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:37.710678 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa715ec-3610-44f2-88cb-8272e2f01dae service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:37.720197 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1488/2945] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:38.093236 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207edeff-3957-4285-9aa4-7387558da826 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:38.094704 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-207edeff-3957-4285-9aa4-7387558da826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:38.108958 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207edeff-3957-4285-9aa4-7387558da826 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:38.113826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6466726-b16d-4018-83fc-2f1c1e9e1226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:38.114563 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6466726-b16d-4018-83fc-2f1c1e9e1226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:38.114960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6466726-b16d-4018-83fc-2f1c1e9e1226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:38.115310 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6466726-b16d-4018-83fc-2f1c1e9e1226 admin admin] PATH_INFO: `/v3/projects/11615d7cea1f4a878c5d5be1c78d735f` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:38.116583 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6466726-b16d-4018-83fc-2f1c1e9e1226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11615d7cea1f4a878c5d5be1c78d735f)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:38.118551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6466726-b16d-4018-83fc-2f1c1e9e1226 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:38.211592 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-b6466726-b16d-4018-83fc-2f1c1e9e1226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11615d7cea1f4a878c5d5be1c78d735f', 'request_id': 'req-b6466726-b16d-4018-83fc-2f1c1e9e1226'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:38.217283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-b6466726-b16d-4018-83fc-2f1c1e9e1226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11615d7cea1f4a878c5d5be1c78d735f', 'request_id': 'req-b6466726-b16d-4018-83fc-2f1c1e9e1226'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:38.268503 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-b6466726-b16d-4018-83fc-2f1c1e9e1226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11615d7cea1f4a878c5d5be1c78d735f', 'request_id': 'req-b6466726-b16d-4018-83fc-2f1c1e9e1226'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:38.280477 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-b6466726-b16d-4018-83fc-2f1c1e9e1226 admin admin] The token cache is being invalidate because project 11615d7cea1f4a878c5d5be1c78d735f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:23:38.280765 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-b6466726-b16d-4018-83fc-2f1c1e9e1226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:38.281384 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-b6466726-b16d-4018-83fc-2f1c1e9e1226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:38.282905 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1458/2946] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:23:38 2022] DELETE /identity/v3/projects/11615d7cea1f4a878c5d5be1c78d735f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:38.287518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa715ec-3610-44f2-88cb-8272e2f01dae service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:38.297125 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa715ec-3610-44f2-88cb-8272e2f01dae service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:38.353294 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4e577-ed1d-4bc2-88cd-8ef42cad1a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:38.353921 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4e577-ed1d-4bc2-88cd-8ef42cad1a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:38.354205 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4e577-ed1d-4bc2-88cd-8ef42cad1a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:38.354481 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4e577-ed1d-4bc2-88cd-8ef42cad1a4e admin admin] PATH_INFO: `/v3/users/1b9e2fb7245141328c717eec22279a7e` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:38.355638 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4e577-ed1d-4bc2-88cd-8ef42cad1a4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b9e2fb7245141328c717eec22279a7e)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:38.357586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4e577-ed1d-4bc2-88cd-8ef42cad1a4e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:38.359100 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:38.568160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-05d4e577-ed1d-4bc2-88cd-8ef42cad1a4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b9e2fb7245141328c717eec22279a7e'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:38.634413 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-05d4e577-ed1d-4bc2-88cd-8ef42cad1a4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b9e2fb7245141328c717eec22279a7e'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:38.647909 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-05d4e577-ed1d-4bc2-88cd-8ef42cad1a4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b9e2fb7245141328c717eec22279a7e'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:38.648539 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-05d4e577-ed1d-4bc2-88cd-8ef42cad1a4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b9e2fb7245141328c717eec22279a7e'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:38.655410 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-05d4e577-ed1d-4bc2-88cd-8ef42cad1a4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b9e2fb7245141328c717eec22279a7e'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:38.661169 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1489/2947] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:23:38 2022] DELETE /identity/v3/users/1b9e2fb7245141328c717eec22279a7e => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:39.085254 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6466726-b16d-4018-83fc-2f1c1e9e1226 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:39.087557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b6466726-b16d-4018-83fc-2f1c1e9e1226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:39.104653 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6466726-b16d-4018-83fc-2f1c1e9e1226 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:39.164173 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef506e2d-869f-4032-8fc2-7575fc536eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:39.164814 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef506e2d-869f-4032-8fc2-7575fc536eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:39.165095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef506e2d-869f-4032-8fc2-7575fc536eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:39.165372 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef506e2d-869f-4032-8fc2-7575fc536eff admin admin] PATH_INFO: `/v3/projects/f97dc89cbdd64289a44f7a6dc3b601a0` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:39.166512 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef506e2d-869f-4032-8fc2-7575fc536eff admin admin] RBAC: Authorizing `identity:delete_project(project_id=f97dc89cbdd64289a44f7a6dc3b601a0)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:39.168453 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef506e2d-869f-4032-8fc2-7575fc536eff admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:39.170000 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:39.289749 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-ef506e2d-869f-4032-8fc2-7575fc536eff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f97dc89cbdd64289a44f7a6dc3b601a0', 'request_id': 'req-ef506e2d-869f-4032-8fc2-7575fc536eff'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:39.295195 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-ef506e2d-869f-4032-8fc2-7575fc536eff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f97dc89cbdd64289a44f7a6dc3b601a0', 'request_id': 'req-ef506e2d-869f-4032-8fc2-7575fc536eff'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:39.369155 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-ef506e2d-869f-4032-8fc2-7575fc536eff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f97dc89cbdd64289a44f7a6dc3b601a0', 'request_id': 'req-ef506e2d-869f-4032-8fc2-7575fc536eff'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:39.380260 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-ef506e2d-869f-4032-8fc2-7575fc536eff admin admin] The token cache is being invalidate because project f97dc89cbdd64289a44f7a6dc3b601a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:23:39.380542 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-ef506e2d-869f-4032-8fc2-7575fc536eff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:39.381163 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-ef506e2d-869f-4032-8fc2-7575fc536eff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:39.382621 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1459/2948] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:23:39 2022] DELETE /identity/v3/projects/f97dc89cbdd64289a44f7a6dc3b601a0 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:39.387520 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d967ee-4e74-4063-8613-853e80c75801 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:39.388000 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d967ee-4e74-4063-8613-853e80c75801 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:39.388093 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d967ee-4e74-4063-8613-853e80c75801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:39.408091 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-57d967ee-4e74-4063-8613-853e80c75801 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:39.409283 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-57d967ee-4e74-4063-8613-853e80c75801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:39.466830 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1490/2949] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:39.475216 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb7512-2264-4222-b6cd-a85548945f0c None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:39.475500 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb7512-2264-4222-b6cd-a85548945f0c None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:39.475773 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb7512-2264-4222-b6cd-a85548945f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:39.495728 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-b3eb7512-2264-4222-b6cd-a85548945f0c None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:39.496910 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b3eb7512-2264-4222-b6cd-a85548945f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:39.504665 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1460/2950] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:39.508778 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d967ee-4e74-4063-8613-853e80c75801 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:39.518523 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d967ee-4e74-4063-8613-853e80c75801 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:39.523547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a810cb-87fe-436b-bc8d-9093273874c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:39.524134 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a810cb-87fe-436b-bc8d-9093273874c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:39.524414 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a810cb-87fe-436b-bc8d-9093273874c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:39.524687 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a810cb-87fe-436b-bc8d-9093273874c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:39.525319 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a810cb-87fe-436b-bc8d-9093273874c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:23:39.525621 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a810cb-87fe-436b-bc8d-9093273874c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:39.527249 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a810cb-87fe-436b-bc8d-9093273874c4 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:39.539089 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1491/2951] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 15:23:39.543151 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb7512-2264-4222-b6cd-a85548945f0c None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:39.552018 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb7512-2264-4222-b6cd-a85548945f0c None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:39.556575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc392c0-a310-4621-9a94-5eeecaa7f290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:39.557155 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc392c0-a310-4621-9a94-5eeecaa7f290 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:39.557448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc392c0-a310-4621-9a94-5eeecaa7f290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:39.557702 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc392c0-a310-4621-9a94-5eeecaa7f290 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:39.558311 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc392c0-a310-4621-9a94-5eeecaa7f290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:39.560213 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc392c0-a310-4621-9a94-5eeecaa7f290 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:39.648556 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1461/2952] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:23:39 2022] POST /identity/v3/projects => generated 359 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:23:39.652578 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a810cb-87fe-436b-bc8d-9093273874c4 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:39.663033 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a810cb-87fe-436b-bc8d-9093273874c4 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:39.720316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299a622-515a-42e5-b014-19bcbb7cdba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:39.720924 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299a622-515a-42e5-b014-19bcbb7cdba2 admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:39.721204 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299a622-515a-42e5-b014-19bcbb7cdba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:39.721476 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299a622-515a-42e5-b014-19bcbb7cdba2 admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:39.722117 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299a622-515a-42e5-b014-19bcbb7cdba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:39.724790 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299a622-515a-42e5-b014-19bcbb7cdba2 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:39.735483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:39.820328 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1492/2953] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:23:39 2022] POST /identity/v3/users => generated 280 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:23:39.824423 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc392c0-a310-4621-9a94-5eeecaa7f290 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:39.835488 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc392c0-a310-4621-9a94-5eeecaa7f290 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:39.840163 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bb1e8-b645-4d8f-8800-ed5dd9d3ed7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:39.840752 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb1e8-b645-4d8f-8800-ed5dd9d3ed7f admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:39.841051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb1e8-b645-4d8f-8800-ed5dd9d3ed7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:39.841294 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb1e8-b645-4d8f-8800-ed5dd9d3ed7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:39.841926 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bb1e8-b645-4d8f-8800-ed5dd9d3ed7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:39.843579 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bb1e8-b645-4d8f-8800-ed5dd9d3ed7f admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:39.855529 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1462/2954] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:23:39.858258 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299a622-515a-42e5-b014-19bcbb7cdba2 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:39.869022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299a622-515a-42e5-b014-19bcbb7cdba2 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:39.874242 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33aca5a-e079-4d6a-b065-716ac13c4da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:39.874363 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33aca5a-e079-4d6a-b065-716ac13c4da2 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:39.874574 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33aca5a-e079-4d6a-b065-716ac13c4da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:39.874892 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33aca5a-e079-4d6a-b065-716ac13c4da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:39.875534 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33aca5a-e079-4d6a-b065-716ac13c4da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:39.877161 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33aca5a-e079-4d6a-b065-716ac13c4da2 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:39.888146 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1493/2955] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:23:39.892624 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bb1e8-b645-4d8f-8800-ed5dd9d3ed7f admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:39.908830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bb1e8-b645-4d8f-8800-ed5dd9d3ed7f admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:39.908830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803e4cd-4c9a-4798-96e1-8052016771ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:39.908830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803e4cd-4c9a-4798-96e1-8052016771ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:39.909157 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803e4cd-4c9a-4798-96e1-8052016771ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:39.909254 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803e4cd-4c9a-4798-96e1-8052016771ea admin admin] PATH_INFO: `/v3/projects/4770e6d7f267435cb9dd8ecb5d6c8455/users/666208e6d2fa406581813f0f50fcbb7b/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:39.923641 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803e4cd-4c9a-4798-96e1-8052016771ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=4770e6d7f267435cb9dd8ecb5d6c8455, user_id=666208e6d2fa406581813f0f50fcbb7b, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:39.926293 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803e4cd-4c9a-4798-96e1-8052016771ea admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:40.012155 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1463/2956] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:23:39 2022] PUT /identity/v3/projects/4770e6d7f267435cb9dd8ecb5d6c8455/users/666208e6d2fa406581813f0f50fcbb7b/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:23:40.024282 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdd469-e0ff-46b7-a64b-a95326ce7209 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:40.029083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdd469-e0ff-46b7-a64b-a95326ce7209 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:40.029083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdd469-e0ff-46b7-a64b-a95326ce7209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:40.052307 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-94bdd469-e0ff-46b7-a64b-a95326ce7209 None None] MFA Rules not processed for user `666208e6d2fa406581813f0f50fcbb7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:40.053519 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-94bdd469-e0ff-46b7-a64b-a95326ce7209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:40.118084 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1494/2957] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:40 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:40.150843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803e4cd-4c9a-4798-96e1-8052016771ea admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:40.152647 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b803e4cd-4c9a-4798-96e1-8052016771ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:40.209370 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803e4cd-4c9a-4798-96e1-8052016771ea admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:40.212968 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367f431-3434-4476-b32e-7a8808f1f014 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:40.213569 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367f431-3434-4476-b32e-7a8808f1f014 service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:40.213852 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367f431-3434-4476-b32e-7a8808f1f014 service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:40.214129 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367f431-3434-4476-b32e-7a8808f1f014 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:40.222262 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a367f431-3434-4476-b32e-7a8808f1f014 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:40.224223 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a367f431-3434-4476-b32e-7a8808f1f014 service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:40.232882 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1464/2958] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:40 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:40.249361 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdd469-e0ff-46b7-a64b-a95326ce7209 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:40.311088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdd469-e0ff-46b7-a64b-a95326ce7209 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:40.315653 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764eeab-f814-4cb3-ae67-f96b8e199af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:40.316271 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764eeab-f814-4cb3-ae67-f96b8e199af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:40.316548 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764eeab-f814-4cb3-ae67-f96b8e199af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:40.316819 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764eeab-f814-4cb3-ae67-f96b8e199af0 admin admin] PATH_INFO: `/v3/users/666208e6d2fa406581813f0f50fcbb7b` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:40.317931 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764eeab-f814-4cb3-ae67-f96b8e199af0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=666208e6d2fa406581813f0f50fcbb7b)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:40.319871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764eeab-f814-4cb3-ae67-f96b8e199af0 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:40.476373 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8764eeab-f814-4cb3-ae67-f96b8e199af0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '666208e6d2fa406581813f0f50fcbb7b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:40.531351 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8764eeab-f814-4cb3-ae67-f96b8e199af0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '666208e6d2fa406581813f0f50fcbb7b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:40.543911 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8764eeab-f814-4cb3-ae67-f96b8e199af0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '666208e6d2fa406581813f0f50fcbb7b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:40.544593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8764eeab-f814-4cb3-ae67-f96b8e199af0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '666208e6d2fa406581813f0f50fcbb7b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:40.548993 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:40.551816 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-8764eeab-f814-4cb3-ae67-f96b8e199af0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '666208e6d2fa406581813f0f50fcbb7b'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:40.557518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1495/2959] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:23:40 2022] DELETE /identity/v3/users/666208e6d2fa406581813f0f50fcbb7b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:23:40.561997 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1ed0c-0d9e-4891-8f81-6905b9c5f32b None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:40.562280 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1ed0c-0d9e-4891-8f81-6905b9c5f32b None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:40.562555 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1ed0c-0d9e-4891-8f81-6905b9c5f32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:40.581148 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-26f1ed0c-0d9e-4891-8f81-6905b9c5f32b None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:40.583349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-26f1ed0c-0d9e-4891-8f81-6905b9c5f32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:40.607552 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:40.641312 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1465/2960] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:40.651774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764eeab-f814-4cb3-ae67-f96b8e199af0 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:40.653202 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-8764eeab-f814-4cb3-ae67-f96b8e199af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:40.710025 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764eeab-f814-4cb3-ae67-f96b8e199af0 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:40.713793 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37530808-6f76-49e6-8bf6-4b4163b107e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:40.714404 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37530808-6f76-49e6-8bf6-4b4163b107e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:40.714686 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37530808-6f76-49e6-8bf6-4b4163b107e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:40.714996 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37530808-6f76-49e6-8bf6-4b4163b107e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:40.724111 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37530808-6f76-49e6-8bf6-4b4163b107e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:40.725591 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37530808-6f76-49e6-8bf6-4b4163b107e1 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:40.735147 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1496/2961] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:41.128218 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1ed0c-0d9e-4891-8f81-6905b9c5f32b None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:41.132245 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-26f1ed0c-0d9e-4891-8f81-6905b9c5f32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:41.142026 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1ed0c-0d9e-4891-8f81-6905b9c5f32b None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:41.147551 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d2098-fabe-46b7-8113-fb4fe8590482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:41.148194 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d2098-fabe-46b7-8113-fb4fe8590482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:41.148479 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d2098-fabe-46b7-8113-fb4fe8590482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:41.148757 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d2098-fabe-46b7-8113-fb4fe8590482 admin admin] PATH_INFO: `/v3/projects/4770e6d7f267435cb9dd8ecb5d6c8455` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:41.149876 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d2098-fabe-46b7-8113-fb4fe8590482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4770e6d7f267435cb9dd8ecb5d6c8455)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:41.152524 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d2098-fabe-46b7-8113-fb4fe8590482 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:41.245380 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9e0d2098-fabe-46b7-8113-fb4fe8590482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4770e6d7f267435cb9dd8ecb5d6c8455', 'request_id': 'req-9e0d2098-fabe-46b7-8113-fb4fe8590482'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:41.251095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9e0d2098-fabe-46b7-8113-fb4fe8590482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4770e6d7f267435cb9dd8ecb5d6c8455', 'request_id': 'req-9e0d2098-fabe-46b7-8113-fb4fe8590482'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:41.317422 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9e0d2098-fabe-46b7-8113-fb4fe8590482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4770e6d7f267435cb9dd8ecb5d6c8455', 'request_id': 'req-9e0d2098-fabe-46b7-8113-fb4fe8590482'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:41.328826 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9e0d2098-fabe-46b7-8113-fb4fe8590482 admin admin] The token cache is being invalidate because project 4770e6d7f267435cb9dd8ecb5d6c8455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81594) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:23:41.329119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9e0d2098-fabe-46b7-8113-fb4fe8590482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:41.329810 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-9e0d2098-fabe-46b7-8113-fb4fe8590482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:23:41.331368 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1466/2962] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:23:41 2022] DELETE /identity/v3/projects/4770e6d7f267435cb9dd8ecb5d6c8455 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:41.337975 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47fa21-9e21-4b6b-843e-93b299085316 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:41.338269 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47fa21-9e21-4b6b-843e-93b299085316 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:41.338548 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47fa21-9e21-4b6b-843e-93b299085316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:41.358462 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-0b47fa21-9e21-4b6b-843e-93b299085316 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:41.359702 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-0b47fa21-9e21-4b6b-843e-93b299085316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:41.418221 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1497/2963] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:41.426081 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0730a-61f5-4ee1-8a3b-b6af6ba63b45 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:41.426373 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0730a-61f5-4ee1-8a3b-b6af6ba63b45 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:41.426651 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0730a-61f5-4ee1-8a3b-b6af6ba63b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:41.446353 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-8ac0730a-61f5-4ee1-8a3b-b6af6ba63b45 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:41.447585 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-8ac0730a-61f5-4ee1-8a3b-b6af6ba63b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:41.455376 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1467/2964] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:41.460069 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47fa21-9e21-4b6b-843e-93b299085316 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:41.469483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47fa21-9e21-4b6b-843e-93b299085316 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:41.474016 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f37a83-c743-42cd-8b6b-d7ae7615857c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:41.474601 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f37a83-c743-42cd-8b6b-d7ae7615857c admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:41.474925 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f37a83-c743-42cd-8b6b-d7ae7615857c admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:41.475202 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f37a83-c743-42cd-8b6b-d7ae7615857c admin admin] PATH_INFO: `/v3/domains` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:41.475850 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f37a83-c743-42cd-8b6b-d7ae7615857c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81595) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 15:23:41.476151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f37a83-c743-42cd-8b6b-d7ae7615857c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:41.477724 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f37a83-c743-42cd-8b6b-d7ae7615857c admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:41.488922 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1498/2965] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 5 15:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 15:23:41.492932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0730a-61f5-4ee1-8a3b-b6af6ba63b45 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:41.502039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0730a-61f5-4ee1-8a3b-b6af6ba63b45 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:41.507283 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777461b-5caf-46a2-9c1d-4d583cff1a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:41.507869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777461b-5caf-46a2-9c1d-4d583cff1a0f admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:41.508149 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777461b-5caf-46a2-9c1d-4d583cff1a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:41.508422 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777461b-5caf-46a2-9c1d-4d583cff1a0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:41.509044 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777461b-5caf-46a2-9c1d-4d583cff1a0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:41.510998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777461b-5caf-46a2-9c1d-4d583cff1a0f admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:41.592850 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1468/2966] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:23:41 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:23:41.596907 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f37a83-c743-42cd-8b6b-d7ae7615857c admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:41.606812 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f37a83-c743-42cd-8b6b-d7ae7615857c admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:41.662394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132659db-5ec3-451c-9775-280e7df85f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:41.663027 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132659db-5ec3-451c-9775-280e7df85f1c admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:41.663307 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132659db-5ec3-451c-9775-280e7df85f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:41.663577 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132659db-5ec3-451c-9775-280e7df85f1c admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:41.664199 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132659db-5ec3-451c-9775-280e7df85f1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:41.666093 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132659db-5ec3-451c-9775-280e7df85f1c admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:41.740951 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1499/2967] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:23:41 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:23:41.745090 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777461b-5caf-46a2-9c1d-4d583cff1a0f admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:41.755544 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777461b-5caf-46a2-9c1d-4d583cff1a0f admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:41.760097 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f15ef09-638f-42d3-8014-e6a1e2cf9c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:41.763025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15ef09-638f-42d3-8014-e6a1e2cf9c0b admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:41.763025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15ef09-638f-42d3-8014-e6a1e2cf9c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:41.763025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15ef09-638f-42d3-8014-e6a1e2cf9c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:41.763025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f15ef09-638f-42d3-8014-e6a1e2cf9c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:41.763541 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f15ef09-638f-42d3-8014-e6a1e2cf9c0b admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:41.774060 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1469/2968] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 15:23:41.777815 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132659db-5ec3-451c-9775-280e7df85f1c admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:41.788268 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132659db-5ec3-451c-9775-280e7df85f1c admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:41.792843 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fe196-209d-4d5a-aec5-c24385674c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:41.793423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fe196-209d-4d5a-aec5-c24385674c70 admin admin] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:41.793701 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fe196-209d-4d5a-aec5-c24385674c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:41.793972 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fe196-209d-4d5a-aec5-c24385674c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:41.794630 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fe196-209d-4d5a-aec5-c24385674c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:41.796237 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fe196-209d-4d5a-aec5-c24385674c70 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:41.806595 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1500/2969] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:23:41.810637 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f15ef09-638f-42d3-8014-e6a1e2cf9c0b admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:41.820361 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f15ef09-638f-42d3-8014-e6a1e2cf9c0b admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:41.824983 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026c47b-b6a1-4f23-a4bd-4feef2ebcc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:41.825617 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026c47b-b6a1-4f23-a4bd-4feef2ebcc91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:41.825902 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026c47b-b6a1-4f23-a4bd-4feef2ebcc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:41.826179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026c47b-b6a1-4f23-a4bd-4feef2ebcc91 admin admin] PATH_INFO: `/v3/projects/849d1e5bc370483790ad5d08840aee6b/users/4c108d58bd84404584e02db115dd8cd5/roles/5812e6538f3f4bdfbfbe11cd25c02957` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:41.841981 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b026c47b-b6a1-4f23-a4bd-4feef2ebcc91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=849d1e5bc370483790ad5d08840aee6b, user_id=4c108d58bd84404584e02db115dd8cd5, role_id=5812e6538f3f4bdfbfbe11cd25c02957)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:41.844663 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b026c47b-b6a1-4f23-a4bd-4feef2ebcc91 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:41.914765 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1470/2970] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:23:41 2022] PUT /identity/v3/projects/849d1e5bc370483790ad5d08840aee6b/users/4c108d58bd84404584e02db115dd8cd5/roles/5812e6538f3f4bdfbfbe11cd25c02957 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:23:41.920085 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fdd31-2e4b-4bb5-9923-3668d5ec46b9 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:41.920374 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fdd31-2e4b-4bb5-9923-3668d5ec46b9 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:41.920696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fdd31-2e4b-4bb5-9923-3668d5ec46b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:41.928152 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:41.940552 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-d61fdd31-2e4b-4bb5-9923-3668d5ec46b9 None None] MFA Rules not processed for user `40eedd635c074248bf2c6c67ebf16bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:41.941820 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-d61fdd31-2e4b-4bb5-9923-3668d5ec46b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:42.000051 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1501/2971] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:42.009791 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026c47b-b6a1-4f23-a4bd-4feef2ebcc91 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:42.011422 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b026c47b-b6a1-4f23-a4bd-4feef2ebcc91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:42.063630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:42.067387 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026c47b-b6a1-4f23-a4bd-4feef2ebcc91 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:42.071003 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705a866-3cbf-480c-8f5a-d05d7c1be9ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:42.071606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705a866-3cbf-480c-8f5a-d05d7c1be9ba service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:42.071881 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705a866-3cbf-480c-8f5a-d05d7c1be9ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:42.072150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705a866-3cbf-480c-8f5a-d05d7c1be9ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:42.081201 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705a866-3cbf-480c-8f5a-d05d7c1be9ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:42.082629 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705a866-3cbf-480c-8f5a-d05d7c1be9ba service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:42.092379 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1471/2972] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:23:47.340102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b842a7d-a04f-4e67-9aa7-211f49c02311 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:47.340102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b842a7d-a04f-4e67-9aa7-211f49c02311 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:47.340102 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b842a7d-a04f-4e67-9aa7-211f49c02311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:47.393566 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-7b842a7d-a04f-4e67-9aa7-211f49c02311 None None] MFA Rules not processed for user `4c108d58bd84404584e02db115dd8cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:47.394761 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-7b842a7d-a04f-4e67-9aa7-211f49c02311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:47.465367 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1502/2973] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:47.469830 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705a866-3cbf-480c-8f5a-d05d7c1be9ba service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:47.480025 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705a866-3cbf-480c-8f5a-d05d7c1be9ba service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:47.484774 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41175bfc-b1ee-4b2a-91f8-9706499df597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:47.485383 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41175bfc-b1ee-4b2a-91f8-9706499df597 admin admin] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:47.485665 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41175bfc-b1ee-4b2a-91f8-9706499df597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:47.485936 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41175bfc-b1ee-4b2a-91f8-9706499df597 admin admin] PATH_INFO: `/v3/projects` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:47.486575 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41175bfc-b1ee-4b2a-91f8-9706499df597 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:47.488618 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41175bfc-b1ee-4b2a-91f8-9706499df597 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:47.571599 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1472/2974] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 5 15:23:47 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 15:23:47.575931 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b842a7d-a04f-4e67-9aa7-211f49c02311 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:47.587602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b842a7d-a04f-4e67-9aa7-211f49c02311 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:47.648187 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cb72c-8534-4369-94b5-f42d18edf73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:47.648798 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cb72c-8534-4369-94b5-f42d18edf73e admin admin] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:47.649083 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cb72c-8534-4369-94b5-f42d18edf73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:47.649356 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cb72c-8534-4369-94b5-f42d18edf73e admin admin] PATH_INFO: `/v3/users` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:47.649988 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11cb72c-8534-4369-94b5-f42d18edf73e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:47.652325 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11cb72c-8534-4369-94b5-f42d18edf73e admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:47.741352 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1503/2975] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 5 15:23:47 2022] POST /identity/v3/users => generated 296 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 15:23:47.745506 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41175bfc-b1ee-4b2a-91f8-9706499df597 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:47.757299 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41175bfc-b1ee-4b2a-91f8-9706499df597 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:47.762100 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e991650-3ae6-43b9-9201-30f0819ed7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:47.764554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e991650-3ae6-43b9-9201-30f0819ed7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:47.764554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e991650-3ae6-43b9-9201-30f0819ed7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:47.764554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e991650-3ae6-43b9-9201-30f0819ed7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:47.764554 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e991650-3ae6-43b9-9201-30f0819ed7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:47.765859 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e991650-3ae6-43b9-9201-30f0819ed7c3 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:47.778817 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1473/2976] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 15:23:47.783072 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cb72c-8534-4369-94b5-f42d18edf73e admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:47.800405 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cb72c-8534-4369-94b5-f42d18edf73e admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:47.806631 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8e691-26ae-46ec-b9e3-257fbf7dbdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:47.807407 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8e691-26ae-46ec-b9e3-257fbf7dbdf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:47.807779 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8e691-26ae-46ec-b9e3-257fbf7dbdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:47.808139 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8e691-26ae-46ec-b9e3-257fbf7dbdf6 admin admin] PATH_INFO: `/v3/projects/164e1e669a3d407ebaa34c5c7b276869/users/b24b1d15301645db9f5a6fcfa28d37a6/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:47.829954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8e691-26ae-46ec-b9e3-257fbf7dbdf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=164e1e669a3d407ebaa34c5c7b276869, user_id=b24b1d15301645db9f5a6fcfa28d37a6, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:47.832696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8e691-26ae-46ec-b9e3-257fbf7dbdf6 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:47.908131 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:23:47.914582 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1504/2977] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:23:47 2022] PUT /identity/v3/projects/164e1e669a3d407ebaa34c5c7b276869/users/b24b1d15301645db9f5a6fcfa28d37a6/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:23:47.918322 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e991650-3ae6-43b9-9201-30f0819ed7c3 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:47.931474 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e991650-3ae6-43b9-9201-30f0819ed7c3 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:47.998894 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741647db-655e-4fc7-85bc-bd61b6d96cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:47.999670 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741647db-655e-4fc7-85bc-bd61b6d96cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:47.999960 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741647db-655e-4fc7-85bc-bd61b6d96cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:48.000242 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741647db-655e-4fc7-85bc-bd61b6d96cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:48.000901 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741647db-655e-4fc7-85bc-bd61b6d96cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:48.002719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741647db-655e-4fc7-85bc-bd61b6d96cd1 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:48.015932 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1474/2978] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 5 15:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 15:23:48.020530 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8e691-26ae-46ec-b9e3-257fbf7dbdf6 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:48.034219 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8e691-26ae-46ec-b9e3-257fbf7dbdf6 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:48.041208 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e506ad-e113-47f0-ad37-8ba7d9b704c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:48.041856 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e506ad-e113-47f0-ad37-8ba7d9b704c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:48.042148 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e506ad-e113-47f0-ad37-8ba7d9b704c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:48.042429 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e506ad-e113-47f0-ad37-8ba7d9b704c6 admin admin] PATH_INFO: `/v3/projects/164e1e669a3d407ebaa34c5c7b276869/users/b24b1d15301645db9f5a6fcfa28d37a6/roles/affac03b860a4fe0b94e52140e78ab79` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:48.044773 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e506ad-e113-47f0-ad37-8ba7d9b704c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=164e1e669a3d407ebaa34c5c7b276869, user_id=b24b1d15301645db9f5a6fcfa28d37a6, role_id=affac03b860a4fe0b94e52140e78ab79)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:48.047488 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e506ad-e113-47f0-ad37-8ba7d9b704c6 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:48.063323 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1505/2979] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 5 15:23:48 2022] PUT /identity/v3/projects/164e1e669a3d407ebaa34c5c7b276869/users/b24b1d15301645db9f5a6fcfa28d37a6/roles/affac03b860a4fe0b94e52140e78ab79 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:23:53.551557 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f05a3-fc87-4df0-bf9e-1fcd075ba907 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:53.552083 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f05a3-fc87-4df0-bf9e-1fcd075ba907 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:53.552164 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f05a3-fc87-4df0-bf9e-1fcd075ba907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:53.586393 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-d82f05a3-fc87-4df0-bf9e-1fcd075ba907 None None] MFA Rules not processed for user `b24b1d15301645db9f5a6fcfa28d37a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:23:53.587848 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d82f05a3-fc87-4df0-bf9e-1fcd075ba907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:53.665408 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1475/2980] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:23:53 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:23:53.682880 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e506ad-e113-47f0-ad37-8ba7d9b704c6 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:23:53.684718 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-49e506ad-e113-47f0-ad37-8ba7d9b704c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:23:53.764871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e506ad-e113-47f0-ad37-8ba7d9b704c6 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:23:53.771071 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1edc19-f9a1-4e4a-b92d-5134c9b85955 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:23:53.771697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1edc19-f9a1-4e4a-b92d-5134c9b85955 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:23:53.771978 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1edc19-f9a1-4e4a-b92d-5134c9b85955 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:23:53.772251 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1edc19-f9a1-4e4a-b92d-5134c9b85955 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:23:53.832077 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1edc19-f9a1-4e4a-b92d-5134c9b85955 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:23:53.834150 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1edc19-f9a1-4e4a-b92d-5134c9b85955 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:23:53.843581 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1506/2981] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:23:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:24:03.662282 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=81,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:24:03.839189 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=62,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:24:09.573580 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f05a3-fc87-4df0-bf9e-1fcd075ba907 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:24:09.575055 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d82f05a3-fc87-4df0-bf9e-1fcd075ba907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:24:09.674452 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f05a3-fc87-4df0-bf9e-1fcd075ba907 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:24:09.679458 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4abb4-123d-465d-86aa-005fc49a45c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:24:09.680052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4abb4-123d-465d-86aa-005fc49a45c8 service placement] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:24:09.680329 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4abb4-123d-465d-86aa-005fc49a45c8 service placement] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:24:09.680599 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4abb4-123d-465d-86aa-005fc49a45c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:24:09.682467 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-5de4abb4-123d-465d-86aa-005fc49a45c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:24:09.693923 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4abb4-123d-465d-86aa-005fc49a45c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:24:09.695573 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4abb4-123d-465d-86aa-005fc49a45c8 service placement] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:24:09.705716 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1476/2982] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:24:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:24:19.701484 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=45 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:24:22.828423 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fbc28-1081-4b4b-9ee8-afe8b1c5c0b9 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:24:22.828937 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fbc28-1081-4b4b-9ee8-afe8b1c5c0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:24:22.829041 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fbc28-1081-4b4b-9ee8-afe8b1c5c0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:24:22.847884 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-506fbc28-1081-4b4b-9ee8-afe8b1c5c0b9 None None] MFA Rules not processed for user `4c108d58bd84404584e02db115dd8cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:24:22.849099 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-506fbc28-1081-4b4b-9ee8-afe8b1c5c0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:24:22.856009 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1507/2983] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:24:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:24:23.167858 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e76bd1-2d6c-479e-a920-6219a57ee527 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:24:23.168375 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e76bd1-2d6c-479e-a920-6219a57ee527 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:24:23.168562 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e76bd1-2d6c-479e-a920-6219a57ee527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:24:23.214430 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-b0e76bd1-2d6c-479e-a920-6219a57ee527 None None] MFA Rules not processed for user `5c73ca2785334fc496b5227d3be655c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:24:23.215717 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b0e76bd1-2d6c-479e-a920-6219a57ee527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:24:23.265012 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1477/2984] 10.222.0.15 () {58 vars in 1062 bytes} [Mon Sep 5 15:24:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:24:23.271080 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fbc28-1081-4b4b-9ee8-afe8b1c5c0b9 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:24:23.280440 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fbc28-1081-4b4b-9ee8-afe8b1c5c0b9 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:24:23.283991 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a7a66-ded9-4fe9-bc56-3267268c555b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:24:23.284584 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a7a66-ded9-4fe9-bc56-3267268c555b service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:24:23.284860 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a7a66-ded9-4fe9-bc56-3267268c555b service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:24:23.285128 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a7a66-ded9-4fe9-bc56-3267268c555b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:24:23.286858 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-795a7a66-ded9-4fe9-bc56-3267268c555b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:24:23.296730 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a7a66-ded9-4fe9-bc56-3267268c555b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:24:23.298140 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a7a66-ded9-4fe9-bc56-3267268c555b service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:24:23.306107 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1508/2985] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:24:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:24:24.031982 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e76bd1-2d6c-479e-a920-6219a57ee527 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:24:24.045125 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e76bd1-2d6c-479e-a920-6219a57ee527 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:24:24.050224 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e7cce-b713-4b5f-8a55-1ce7ad681701 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:24:24.050869 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e7cce-b713-4b5f-8a55-1ce7ad681701 service placement] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:24:24.051160 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e7cce-b713-4b5f-8a55-1ce7ad681701 service placement] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:24:24.051447 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e7cce-b713-4b5f-8a55-1ce7ad681701 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:24:24.053226 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-387e7cce-b713-4b5f-8a55-1ce7ad681701 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:24:24.066099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e7cce-b713-4b5f-8a55-1ce7ad681701 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:24:24.068185 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e7cce-b713-4b5f-8a55-1ce7ad681701 service placement] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:24:24.084057 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1478/2986] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:24:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:24:27.254394 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb872f-bcfe-4656-9749-dec7f62a8482 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:24:27.254898 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb872f-bcfe-4656-9749-dec7f62a8482 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:24:27.255004 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb872f-bcfe-4656-9749-dec7f62a8482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:24:27.274742 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-1cbb872f-bcfe-4656-9749-dec7f62a8482 None None] MFA Rules not processed for user `5c73ca2785334fc496b5227d3be655c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:24:27.275982 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-1cbb872f-bcfe-4656-9749-dec7f62a8482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:24:27.282594 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1509/2987] 10.222.0.15 () {58 vars in 1062 bytes} [Mon Sep 5 15:24:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:24:27.288788 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e7cce-b713-4b5f-8a55-1ce7ad681701 service placement] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:24:27.298465 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e7cce-b713-4b5f-8a55-1ce7ad681701 service placement] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:24:27.302139 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe2b8b-6132-4fe9-b9c3-c9a256a2ad34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:24:27.302739 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe2b8b-6132-4fe9-b9c3-c9a256a2ad34 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:24:27.303057 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe2b8b-6132-4fe9-b9c3-c9a256a2ad34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:24:27.303336 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe2b8b-6132-4fe9-b9c3-c9a256a2ad34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:24:27.305078 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-cdbe2b8b-6132-4fe9-b9c3-c9a256a2ad34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:24:27.316150 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe2b8b-6132-4fe9-b9c3-c9a256a2ad34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:24:27.317698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe2b8b-6132-4fe9-b9c3-c9a256a2ad34 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:24:27.326442 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1479/2988] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:24:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 15:24:37.269632 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=12 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:24:37.322339 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=45 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:24:38.363182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb872f-bcfe-4656-9749-dec7f62a8482 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:24:38.376215 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb872f-bcfe-4656-9749-dec7f62a8482 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:24:38.381617 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe791c6-38ae-428f-a3f1-d26f2a71ed3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:24:38.382214 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe791c6-38ae-428f-a3f1-d26f2a71ed3e service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:24:38.382499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe791c6-38ae-428f-a3f1-d26f2a71ed3e service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:24:38.382772 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe791c6-38ae-428f-a3f1-d26f2a71ed3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:24:38.384496 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-ffe791c6-38ae-428f-a3f1-d26f2a71ed3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:24:38.397289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe791c6-38ae-428f-a3f1-d26f2a71ed3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:24:38.398871 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe791c6-38ae-428f-a3f1-d26f2a71ed3e service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:24:38.410641 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1510/2989] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:24:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:24:41.897076 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe2b8b-6132-4fe9-b9c3-c9a256a2ad34 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:24:41.905961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe2b8b-6132-4fe9-b9c3-c9a256a2ad34 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:24:41.909524 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142aa82-7c49-4704-b59c-d09923d52836 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:24:41.910111 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142aa82-7c49-4704-b59c-d09923d52836 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:24:41.910388 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142aa82-7c49-4704-b59c-d09923d52836 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:24:41.910659 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142aa82-7c49-4704-b59c-d09923d52836 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:24:41.919252 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142aa82-7c49-4704-b59c-d09923d52836 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:24:41.920720 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142aa82-7c49-4704-b59c-d09923d52836 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:24:41.929842 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1480/2990] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:24:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:24:48.405799 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=6 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:24:51.925812 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=6 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:27:00.896981 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe791c6-38ae-428f-a3f1-d26f2a71ed3e service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:27:00.944287 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe791c6-38ae-428f-a3f1-d26f2a71ed3e service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:27:00.951593 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067c4bc-43ec-4909-9328-91d6bf060e4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:27:00.952966 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067c4bc-43ec-4909-9328-91d6bf060e4d service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:27:00.952966 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067c4bc-43ec-4909-9328-91d6bf060e4d service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:27:00.952966 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067c4bc-43ec-4909-9328-91d6bf060e4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:27:00.955236 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-5067c4bc-43ec-4909-9328-91d6bf060e4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:27:00.982433 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067c4bc-43ec-4909-9328-91d6bf060e4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:27:00.984289 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067c4bc-43ec-4909-9328-91d6bf060e4d service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:27:00.996931 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1511/2991] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:27:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:27:02.846762 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670787d-eee9-447f-a9e7-a3187d06daba None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:27:02.847293 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670787d-eee9-447f-a9e7-a3187d06daba None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:27:02.847398 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670787d-eee9-447f-a9e7-a3187d06daba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:27:02.867004 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-c670787d-eee9-447f-a9e7-a3187d06daba None None] MFA Rules not processed for user `4c108d58bd84404584e02db115dd8cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:27:02.868266 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c670787d-eee9-447f-a9e7-a3187d06daba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:27:02.875130 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1481/2992] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:27:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:27:03.868510 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bdb1c-2e35-4b3b-a956-20614700ac30 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:27:03.868979 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bdb1c-2e35-4b3b-a956-20614700ac30 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:27:03.869203 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bdb1c-2e35-4b3b-a956-20614700ac30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:27:03.909151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-4d2bdb1c-2e35-4b3b-a956-20614700ac30 None None] MFA Rules not processed for user `267825dfb0324be0910203cd5abab377`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:27:03.910518 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-4d2bdb1c-2e35-4b3b-a956-20614700ac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:27:03.919603 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1512/2993] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 5 15:27:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:27:03.931392 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670787d-eee9-447f-a9e7-a3187d06daba None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:27:03.932820 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-c670787d-eee9-447f-a9e7-a3187d06daba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:27:03.947825 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670787d-eee9-447f-a9e7-a3187d06daba None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:27:03.952648 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b016126-1eec-4443-bef5-3d026b599360 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:27:03.953237 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b016126-1eec-4443-bef5-3d026b599360 service placement] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:27:03.953515 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b016126-1eec-4443-bef5-3d026b599360 service placement] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:27:03.953793 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b016126-1eec-4443-bef5-3d026b599360 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:27:03.962630 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b016126-1eec-4443-bef5-3d026b599360 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:27:03.964439 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b016126-1eec-4443-bef5-3d026b599360 service placement] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:27:03.974250 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1482/2994] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:27:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:27:13.906451 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=34 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:27:13.970212 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=9 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:27:23.611697 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bdb1c-2e35-4b3b-a956-20614700ac30 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:27:23.624780 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bdb1c-2e35-4b3b-a956-20614700ac30 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:27:23.630248 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52998c1a-81a9-4e10-b07c-8d89992a8049 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:27:23.630885 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52998c1a-81a9-4e10-b07c-8d89992a8049 service nova] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:27:23.631170 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52998c1a-81a9-4e10-b07c-8d89992a8049 service nova] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:27:23.631446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52998c1a-81a9-4e10-b07c-8d89992a8049 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:27:23.633876 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-52998c1a-81a9-4e10-b07c-8d89992a8049 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:27:23.646517 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52998c1a-81a9-4e10-b07c-8d89992a8049 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:27:23.648182 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52998c1a-81a9-4e10-b07c-8d89992a8049 service nova] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:27:23.658999 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1513/2995] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:27:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:27:33.654407 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=6 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:29:13.120054 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6a44e-4fe9-455a-9806-769164c881f2 None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:29:13.120489 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6a44e-4fe9-455a-9806-769164c881f2 None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:29:13.120606 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6a44e-4fe9-455a-9806-769164c881f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:29:13.169710 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-d0d6a44e-4fe9-455a-9806-769164c881f2 None None] MFA Rules not processed for user `e49ac59302cb42f7942954d3859d3675`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:29:13.170916 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d0d6a44e-4fe9-455a-9806-769164c881f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:29:13.228977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1483/2996] 10.222.0.15 () {58 vars in 1061 bytes} [Mon Sep 5 15:29:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:29:13.235712 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52998c1a-81a9-4e10-b07c-8d89992a8049 service nova] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:29:13.244823 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52998c1a-81a9-4e10-b07c-8d89992a8049 service nova] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:29:13.248490 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0641579-3f9e-4d99-831a-692447b2e441 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:29:13.249088 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0641579-3f9e-4d99-831a-692447b2e441 service cinder] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:29:13.249374 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0641579-3f9e-4d99-831a-692447b2e441 service cinder] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:29:13.250285 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0641579-3f9e-4d99-831a-692447b2e441 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:29:13.258351 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0641579-3f9e-4d99-831a-692447b2e441 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:29:13.259967 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0641579-3f9e-4d99-831a-692447b2e441 service cinder] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:29:13.268966 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1514/2997] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:29:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:29:18.318487 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6a44e-4fe9-455a-9806-769164c881f2 None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:29:18.330275 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6a44e-4fe9-455a-9806-769164c881f2 None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:29:18.335119 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879d0bd-f10b-404f-870e-fc2d7d0ff67e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:29:18.335709 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879d0bd-f10b-404f-870e-fc2d7d0ff67e service placement] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:29:18.335984 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879d0bd-f10b-404f-870e-fc2d7d0ff67e service placement] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:29:18.336253 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879d0bd-f10b-404f-870e-fc2d7d0ff67e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:29:18.344449 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879d0bd-f10b-404f-870e-fc2d7d0ff67e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:29:18.346021 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879d0bd-f10b-404f-870e-fc2d7d0ff67e service placement] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:29:18.355952 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1484/2998] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:29:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:29:23.264802 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=6 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:29:25.028144 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3645ef-11ed-4c85-8836-d1b11121afbf None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:29:25.028705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3645ef-11ed-4c85-8836-d1b11121afbf None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:29:25.028705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3645ef-11ed-4c85-8836-d1b11121afbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:29:25.047416 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-3e3645ef-11ed-4c85-8836-d1b11121afbf None None] MFA Rules not processed for user `4c108d58bd84404584e02db115dd8cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:29:25.049120 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-3e3645ef-11ed-4c85-8836-d1b11121afbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:29:25.055895 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1515/2999] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 5 15:29:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:29:25.097274 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879d0bd-f10b-404f-870e-fc2d7d0ff67e service placement] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:29:25.106671 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879d0bd-f10b-404f-870e-fc2d7d0ff67e service placement] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:29:25.110896 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9193aaa-90d8-473f-8940-9ce34c2e3d9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49ac59302cb42f7942954d3859d3675', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:29:25.111540 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9193aaa-90d8-473f-8940-9ce34c2e3d9e service cinder] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:29:25.111824 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9193aaa-90d8-473f-8940-9ce34c2e3d9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:29:25.112099 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9193aaa-90d8-473f-8940-9ce34c2e3d9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:29:25.113836 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-e9193aaa-90d8-473f-8940-9ce34c2e3d9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:29:25.140804 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9193aaa-90d8-473f-8940-9ce34c2e3d9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:29:25.142244 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9193aaa-90d8-473f-8940-9ce34c2e3d9e service cinder] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:29:25.153501 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1485/3000] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:29:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:29:25.443616 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3645ef-11ed-4c85-8836-d1b11121afbf None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:29:25.462710 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3645ef-11ed-4c85-8836-d1b11121afbf None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:29:25.466228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0c692-6970-40f4-8b1f-8f8b7aad8892 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:29:25.466838 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0c692-6970-40f4-8b1f-8f8b7aad8892 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:29:25.467121 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0c692-6970-40f4-8b1f-8f8b7aad8892 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:29:25.467392 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0c692-6970-40f4-8b1f-8f8b7aad8892 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:29:25.474477 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0c692-6970-40f4-8b1f-8f8b7aad8892 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:29:25.475916 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0c692-6970-40f4-8b1f-8f8b7aad8892 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:29:25.483690 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1516/3001] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:29:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:29:26.223373 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9193aaa-90d8-473f-8940-9ce34c2e3d9e service cinder] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:29:26.232533 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9193aaa-90d8-473f-8940-9ce34c2e3d9e service cinder] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:29:26.237095 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf3c70-72c7-4580-88de-87e25d6bb4ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:29:26.237698 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf3c70-72c7-4580-88de-87e25d6bb4ed service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:29:26.237981 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf3c70-72c7-4580-88de-87e25d6bb4ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:29:26.238254 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf3c70-72c7-4580-88de-87e25d6bb4ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:29:26.240039 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-87cf3c70-72c7-4580-88de-87e25d6bb4ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:29:26.253494 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf3c70-72c7-4580-88de-87e25d6bb4ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:29:26.255033 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf3c70-72c7-4580-88de-87e25d6bb4ed service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:29:26.265861 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1486/3002] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:29:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:29:27.213390 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d6b68-6faf-4a9d-8b6d-bcff7cd24a5f None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:29:27.213849 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d6b68-6faf-4a9d-8b6d-bcff7cd24a5f None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:29:27.213954 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d6b68-6faf-4a9d-8b6d-bcff7cd24a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:29:27.245022 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-299d6b68-6faf-4a9d-8b6d-bcff7cd24a5f None None] MFA Rules not processed for user `267825dfb0324be0910203cd5abab377`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:29:27.246254 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-299d6b68-6faf-4a9d-8b6d-bcff7cd24a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:29:27.255499 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1517/3003] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 5 15:29:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:29:27.267005 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf3c70-72c7-4580-88de-87e25d6bb4ed service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:29:27.278506 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf3c70-72c7-4580-88de-87e25d6bb4ed service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:29:27.283568 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bf799-3b69-4ddc-8cf3-63a6215845be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:29:27.284162 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bf799-3b69-4ddc-8cf3-63a6215845be service placement] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:29:27.284443 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bf799-3b69-4ddc-8cf3-63a6215845be service placement] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:29:27.284719 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bf799-3b69-4ddc-8cf3-63a6215845be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:29:27.292877 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bf799-3b69-4ddc-8cf3-63a6215845be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:29:27.294445 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bf799-3b69-4ddc-8cf3-63a6215845be service placement] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:29:27.304425 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1487/3004] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:29:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:29:32.197056 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d6b68-6faf-4a9d-8b6d-bcff7cd24a5f None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:29:32.206431 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d6b68-6faf-4a9d-8b6d-bcff7cd24a5f None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:29:32.210173 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f0779-f42a-486a-883a-70089f2dbf45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:29:32.212765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f0779-f42a-486a-883a-70089f2dbf45 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:29:32.212765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f0779-f42a-486a-883a-70089f2dbf45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:29:32.212765 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f0779-f42a-486a-883a-70089f2dbf45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:29:32.218602 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f0779-f42a-486a-883a-70089f2dbf45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:29:32.220586 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f0779-f42a-486a-883a-70089f2dbf45 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:29:32.228610 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1518/3005] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:29:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:29:37.300474 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=67 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:29:42.224913 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=27 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:29:45.803411 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bf799-3b69-4ddc-8cf3-63a6215845be service placement] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:29:45.805089 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-d64bf799-3b69-4ddc-8cf3-63a6215845be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:29:45.821048 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bf799-3b69-4ddc-8cf3-63a6215845be service placement] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:29:45.826829 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4953145-e4a1-4fcd-9bb9-574b509f2581 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '709b9ae82dcd47408fc9a6d54a54428a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:29:45.827450 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4953145-e4a1-4fcd-9bb9-574b509f2581 service nova] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:29:45.827803 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4953145-e4a1-4fcd-9bb9-574b509f2581 service nova] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:29:45.828156 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4953145-e4a1-4fcd-9bb9-574b509f2581 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:29:45.838333 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4953145-e4a1-4fcd-9bb9-574b509f2581 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:29:45.842856 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4953145-e4a1-4fcd-9bb9-574b509f2581 service nova] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:29:45.851574 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1488/3006] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:29:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:29:48.389562 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f0779-f42a-486a-883a-70089f2dbf45 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:29:48.398807 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f0779-f42a-486a-883a-70089f2dbf45 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:29:48.402382 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd0123-f102-4055-92e5-9c7d0d7cf304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:29:48.402992 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd0123-f102-4055-92e5-9c7d0d7cf304 service neutron] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:29:48.403272 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd0123-f102-4055-92e5-9c7d0d7cf304 service neutron] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:29:48.403546 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd0123-f102-4055-92e5-9c7d0d7cf304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:29:48.413284 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd0123-f102-4055-92e5-9c7d0d7cf304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:29:48.414700 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd0123-f102-4055-92e5-9c7d0d7cf304 service neutron] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:29:48.424354 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1519/3007] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:29:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:29:55.846954 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=6 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:29:58.419859 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=6 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:30:18.556126 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8234d-ea7e-4e31-8e95-0bcc6b677e4d None None] REQUEST_METHOD: `POST` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:30:18.556660 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8234d-ea7e-4e31-8e95-0bcc6b677e4d None None] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:30:18.556765 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8234d-ea7e-4e31-8e95-0bcc6b677e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:30:18.575930 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.auth.core [None req-65b8234d-ea7e-4e31-8e95-0bcc6b677e4d None None] MFA Rules not processed for user `267825dfb0324be0910203cd5abab377`. Rule list: `[]` (Enabled: `True`). {{(pid=81594) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:30:18.577171 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-65b8234d-ea7e-4e31-8e95-0bcc6b677e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:30:18.585787 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1489/3008] 10.222.0.50 () {58 vars in 1015 bytes} [Mon Sep 5 15:30:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:30:18.618431 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd0123-f102-4055-92e5-9c7d0d7cf304 service neutron] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:30:18.629852 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd0123-f102-4055-92e5-9c7d0d7cf304 service neutron] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:30:18.634844 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78497f-829c-4baa-adc4-94870f1dac68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:30:18.636248 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78497f-829c-4baa-adc4-94870f1dac68 service placement] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:30:18.636248 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78497f-829c-4baa-adc4-94870f1dac68 service placement] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:30:18.636248 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78497f-829c-4baa-adc4-94870f1dac68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:30:18.644291 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78497f-829c-4baa-adc4-94870f1dac68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:30:18.646136 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78497f-829c-4baa-adc4-94870f1dac68 service placement] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:30:18.655800 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1520/3009] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:30:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:30:28.571182 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=3 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:30:28.651705 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=6 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:31:56.001961 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8234d-ea7e-4e31-8e95-0bcc6b677e4d None None] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:31:56.011220 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8234d-ea7e-4e31-8e95-0bcc6b677e4d None None] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:31:56.014934 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd14ef1-86fb-4ae6-be74-fb74c5791850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c73ca2785334fc496b5227d3be655c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:31:56.018715 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd14ef1-86fb-4ae6-be74-fb74c5791850 service neutron] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:31:56.018715 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd14ef1-86fb-4ae6-be74-fb74c5791850 service neutron] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:31:56.018715 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd14ef1-86fb-4ae6-be74-fb74c5791850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:31:56.075873 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd14ef1-86fb-4ae6-be74-fb74c5791850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:31:56.077317 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd14ef1-86fb-4ae6-be74-fb74c5791850 service neutron] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:31:56.087035 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1490/3010] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:31:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:31:59.533696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699c567-97a5-42f7-b368-bb5eb5ebcac1 None None] REQUEST_METHOD: `POST` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:31:59.533696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699c567-97a5-42f7-b368-bb5eb5ebcac1 None None] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:31:59.533696 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699c567-97a5-42f7-b368-bb5eb5ebcac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:31:59.552998 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.auth.core [None req-2699c567-97a5-42f7-b368-bb5eb5ebcac1 None None] MFA Rules not processed for user `267825dfb0324be0910203cd5abab377`. Rule list: `[]` (Enabled: `True`). {{(pid=81595) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 15:31:59.556512 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-2699c567-97a5-42f7-b368-bb5eb5ebcac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:31:59.563568 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1521/3011] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Sep 5 15:31:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 15:31:59.569597 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd14ef1-86fb-4ae6-be74-fb74c5791850 service neutron] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:31:59.580560 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd14ef1-86fb-4ae6-be74-fb74c5791850 service neutron] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:31:59.585520 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc35e99-d168-4d32-8c34-83e91d5d2b92 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:31:59.586124 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc35e99-d168-4d32-8c34-83e91d5d2b92 service placement] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:31:59.586409 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc35e99-d168-4d32-8c34-83e91d5d2b92 service placement] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:31:59.586683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc35e99-d168-4d32-8c34-83e91d5d2b92 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:31:59.588448 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-7cc35e99-d168-4d32-8c34-83e91d5d2b92 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:31:59.600052 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc35e99-d168-4d32-8c34-83e91d5d2b92 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:31:59.601619 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc35e99-d168-4d32-8c34-83e91d5d2b92 service placement] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:31:59.612051 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1491/3012] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:31:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:32:03.174374 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699c567-97a5-42f7-b368-bb5eb5ebcac1 None None] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:32:03.186228 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699c567-97a5-42f7-b368-bb5eb5ebcac1 None None] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:32:03.192207 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9a24c-65a6-40f7-a565-cdb74bfba12e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:32:03.192806 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9a24c-65a6-40f7-a565-cdb74bfba12e service placement] REQUEST_METHOD: `GET` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:32:03.193204 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9a24c-65a6-40f7-a565-cdb74bfba12e service placement] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:32:03.193507 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9a24c-65a6-40f7-a565-cdb74bfba12e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:32:03.195311 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-c2a9a24c-65a6-40f7-a565-cdb74bfba12e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:32:03.207795 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9a24c-65a6-40f7-a565-cdb74bfba12e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:32:03.209425 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9a24c-65a6-40f7-a565-cdb74bfba12e service placement] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:32:03.220006 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1522/3013] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:32:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 15:32:03.324090 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc35e99-d168-4d32-8c34-83e91d5d2b92 service placement] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:32:03.335190 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc35e99-d168-4d32-8c34-83e91d5d2b92 service placement] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:32:03.341160 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3cc91-6b1c-4cb3-80f8-42edd2a5d9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:32:03.341788 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3cc91-6b1c-4cb3-80f8-42edd2a5d9ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:32:03.342068 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3cc91-6b1c-4cb3-80f8-42edd2a5d9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:32:03.342337 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3cc91-6b1c-4cb3-80f8-42edd2a5d9ee admin admin] PATH_INFO: `/v3/users/4c108d58bd84404584e02db115dd8cd5` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:32:03.343471 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3cc91-6b1c-4cb3-80f8-42edd2a5d9ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c108d58bd84404584e02db115dd8cd5)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:32:03.345489 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3cc91-6b1c-4cb3-80f8-42edd2a5d9ee admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:32:03.536964 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4ff3cc91-6b1c-4cb3-80f8-42edd2a5d9ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c108d58bd84404584e02db115dd8cd5'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:03.602530 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4ff3cc91-6b1c-4cb3-80f8-42edd2a5d9ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c108d58bd84404584e02db115dd8cd5'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:03.603179 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4ff3cc91-6b1c-4cb3-80f8-42edd2a5d9ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c108d58bd84404584e02db115dd8cd5'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:03.616704 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4ff3cc91-6b1c-4cb3-80f8-42edd2a5d9ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c108d58bd84404584e02db115dd8cd5'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:03.623586 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-4ff3cc91-6b1c-4cb3-80f8-42edd2a5d9ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c108d58bd84404584e02db115dd8cd5'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:03.629416 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1492/3014] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:32:03 2022] DELETE /identity/v3/users/4c108d58bd84404584e02db115dd8cd5 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:32:03.859483 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9a24c-65a6-40f7-a565-cdb74bfba12e service placement] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:32:03.870181 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9a24c-65a6-40f7-a565-cdb74bfba12e service placement] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:32:03.927159 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be1372-2989-4162-ab5b-0bd616d3c491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:32:03.927796 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be1372-2989-4162-ab5b-0bd616d3c491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:32:03.928074 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be1372-2989-4162-ab5b-0bd616d3c491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:32:03.928345 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be1372-2989-4162-ab5b-0bd616d3c491 admin admin] PATH_INFO: `/v3/projects/849d1e5bc370483790ad5d08840aee6b` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:32:03.929486 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be1372-2989-4162-ab5b-0bd616d3c491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=849d1e5bc370483790ad5d08840aee6b)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:32:03.931446 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be1372-2989-4162-ab5b-0bd616d3c491 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:32:04.042774 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-88be1372-2989-4162-ab5b-0bd616d3c491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '849d1e5bc370483790ad5d08840aee6b', 'request_id': 'req-88be1372-2989-4162-ab5b-0bd616d3c491'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:04.048935 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-88be1372-2989-4162-ab5b-0bd616d3c491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '849d1e5bc370483790ad5d08840aee6b', 'request_id': 'req-88be1372-2989-4162-ab5b-0bd616d3c491'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:04.117160 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-88be1372-2989-4162-ab5b-0bd616d3c491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '849d1e5bc370483790ad5d08840aee6b', 'request_id': 'req-88be1372-2989-4162-ab5b-0bd616d3c491'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:04.129785 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-88be1372-2989-4162-ab5b-0bd616d3c491 admin admin] The token cache is being invalidate because project 849d1e5bc370483790ad5d08840aee6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:32:04.130208 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-88be1372-2989-4162-ab5b-0bd616d3c491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:04.130956 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-88be1372-2989-4162-ab5b-0bd616d3c491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:04.132626 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1523/3015] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:32:03 2022] DELETE /identity/v3/projects/849d1e5bc370483790ad5d08840aee6b => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 15:32:04.139583 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3cc91-6b1c-4cb3-80f8-42edd2a5d9ee admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:32:04.140989 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-4ff3cc91-6b1c-4cb3-80f8-42edd2a5d9ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:32:04.155519 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3cc91-6b1c-4cb3-80f8-42edd2a5d9ee admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:32:04.204775 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:32:04.219819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d7f52-ba86-4df6-bcbb-6066dc24a3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:32:04.219819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d7f52-ba86-4df6-bcbb-6066dc24a3b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:32:04.219819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d7f52-ba86-4df6-bcbb-6066dc24a3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:32:04.219819 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d7f52-ba86-4df6-bcbb-6066dc24a3b5 admin admin] PATH_INFO: `/v3/users/b24b1d15301645db9f5a6fcfa28d37a6` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:32:04.220349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d7f52-ba86-4df6-bcbb-6066dc24a3b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b24b1d15301645db9f5a6fcfa28d37a6)` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:32:04.220349 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d7f52-ba86-4df6-bcbb-6066dc24a3b5 admin admin] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:32:04.416363 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-6c6d7f52-ba86-4df6-bcbb-6066dc24a3b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b24b1d15301645db9f5a6fcfa28d37a6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:04.482626 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-6c6d7f52-ba86-4df6-bcbb-6066dc24a3b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b24b1d15301645db9f5a6fcfa28d37a6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:04.483377 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-6c6d7f52-ba86-4df6-bcbb-6066dc24a3b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b24b1d15301645db9f5a6fcfa28d37a6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:04.495409 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-6c6d7f52-ba86-4df6-bcbb-6066dc24a3b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b24b1d15301645db9f5a6fcfa28d37a6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:04.502843 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.notifications [None req-6c6d7f52-ba86-4df6-bcbb-6066dc24a3b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b24b1d15301645db9f5a6fcfa28d37a6'} {{(pid=81594) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:04.508998 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1493/3016] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 5 15:32:04 2022] DELETE /identity/v3/users/b24b1d15301645db9f5a6fcfa28d37a6 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:32:04.750739 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be1372-2989-4162-ab5b-0bd616d3c491 admin admin] Authenticating user token {{(pid=81595) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:32:04.752265 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.fernet_utils [None req-88be1372-2989-4162-ab5b-0bd616d3c491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81595) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:32:04.765062 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be1372-2989-4162-ab5b-0bd616d3c491 admin admin] Validating token access rules against request {{(pid=81595) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:32:04.823665 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a9a72-0944-44d7-b0fb-b9275b2fa8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40eedd635c074248bf2c6c67ebf16bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f189779a37444439ac0c4ef7b42cc427', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81595) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:32:04.824316 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a9a72-0944-44d7-b0fb-b9275b2fa8a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:32:04.824597 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a9a72-0944-44d7-b0fb-b9275b2fa8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:32:04.824903 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a9a72-0944-44d7-b0fb-b9275b2fa8a0 admin admin] PATH_INFO: `/v3/projects/164e1e669a3d407ebaa34c5c7b276869` {{(pid=81595) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:32:04.826018 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a9a72-0944-44d7-b0fb-b9275b2fa8a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=164e1e669a3d407ebaa34c5c7b276869)` {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:32:04.827952 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a9a72-0944-44d7-b0fb-b9275b2fa8a0 admin admin] RBAC: Authorization granted {{(pid=81595) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:32:04.839139 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:32:04.937817 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-365a9a72-0944-44d7-b0fb-b9275b2fa8a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '164e1e669a3d407ebaa34c5c7b276869', 'request_id': 'req-365a9a72-0944-44d7-b0fb-b9275b2fa8a0'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:04.944151 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-365a9a72-0944-44d7-b0fb-b9275b2fa8a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '164e1e669a3d407ebaa34c5c7b276869', 'request_id': 'req-365a9a72-0944-44d7-b0fb-b9275b2fa8a0'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:05.048643 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-365a9a72-0944-44d7-b0fb-b9275b2fa8a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '164e1e669a3d407ebaa34c5c7b276869', 'request_id': 'req-365a9a72-0944-44d7-b0fb-b9275b2fa8a0'} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:05.059113 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-365a9a72-0944-44d7-b0fb-b9275b2fa8a0 admin admin] The token cache is being invalidate because project 164e1e669a3d407ebaa34c5c7b276869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81595) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 15:32:05.059406 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-365a9a72-0944-44d7-b0fb-b9275b2fa8a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:05.060035 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG keystone.notifications [None req-365a9a72-0944-44d7-b0fb-b9275b2fa8a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81595) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 15:32:05.061521 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: [pid: 81595|app: 0|req: 1524/3017] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 5 15:32:04 2022] DELETE /identity/v3/projects/164e1e669a3d407ebaa34c5c7b276869 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 15:32:14.505357 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:32:15.117547 dv-cinder-842905-3-iscsi devstack@keystone.service[81595]: DEBUG dbcounter [-] [81595] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81595) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 15:35:22.468159 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d7f52-ba86-4df6-bcbb-6066dc24a3b5 admin admin] Authenticating user token {{(pid=81594) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 15:35:22.469561 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-6c6d7f52-ba86-4df6-bcbb-6066dc24a3b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:35:22.567441 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d7f52-ba86-4df6-bcbb-6066dc24a3b5 admin admin] Validating token access rules against request {{(pid=81594) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 15:35:22.626331 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323df8e-4a5c-4d39-a9a0-a0fbe525a7d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267825dfb0324be0910203cd5abab377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ca42deaed344eea726fe6e32356efc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81594) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 15:35:22.626977 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323df8e-4a5c-4d39-a9a0-a0fbe525a7d9 service placement] REQUEST_METHOD: `GET` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 15:35:22.627262 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323df8e-4a5c-4d39-a9a0-a0fbe525a7d9 service placement] SCRIPT_NAME: `/identity` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 15:35:22.627564 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323df8e-4a5c-4d39-a9a0-a0fbe525a7d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81594) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 15:35:22.629301 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.fernet_utils [None req-b323df8e-4a5c-4d39-a9a0-a0fbe525a7d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81594) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 15:35:22.640112 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323df8e-4a5c-4d39-a9a0-a0fbe525a7d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 15:35:22.641683 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323df8e-4a5c-4d39-a9a0-a0fbe525a7d9 service placement] RBAC: Authorization granted {{(pid=81594) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 15:35:22.651426 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: [pid: 81594|app: 0|req: 1494/3018] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 5 15:35:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 15:35:32.647204 dv-cinder-842905-3-iscsi devstack@keystone.service[81594]: DEBUG dbcounter [-] [81594] Writing DB stats keystone:SELECT=71 {{(pid=81594) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}